The problem#

For a compile-on-save workflow where some computation is done in response to every change to a file, if there may sometimes be many changes close together, it may be wasteful to respond to all of them. This is often handled by debouncing the events: instead of responding to every change, ignore changes that occur too close together in time.

The solution#

watch_todo_debounced.sh is a modification of the watch_todo.sh script from my recent post on converting todo.txt files to HTML. It uses a script I found called debounce.sh to wait until there have been no updates to the todo.txt file for 5 seconds before generating the HTML file. The core logic looks like this:

while true
do
    inotifywait -e close_write "$1" >/dev/null
    echo "TODO file updated."
done | debounce.sh read $delay "./do-something.sh \"$1\""

$delay is the delay in seconds to wait before taking an action. $1 is file to watch for changes on and ./do-something.sh is the script to run on it when it changes.

The details#

The problem#

Sometimes I have a long video that I only want a shorter section of. Maybe it's a TV show that I want to clip a funny scene out of. Or a video of a concert that I want to clip the individual songs out of so I can put them into my music library. But figuring out exactly where to start and end the clip so there's no weird sounds or flashes due to accidentally including the surrounding video takes some care.

The solution#

In order to determine the right clip location, I wanted an easy way to repeatedly watch and listen to the start or end of a proposed clip and get immediate feedback on adjusting its position frame-by-frame.

find_split is a script that implements this workflow. You give it a video, a proposed frame index to split the video on, a preview context length in number of frames, and whether the preview should be the video before (to the left of) or after (to the right of) the split. It plays the preview window ending or starting at the proposed split and waits for the user to press a key indicating how to move the proposed split and then repeats, starting by playing that preview. Once you've confirmed you've determined the desired frame index, press ` (or just Ctrl+c) to exit. The full keyboard controls are described at the link.

$ ./find_split video.mkv 1000 60 left
Playing video.mkv on left of split at 1000 \(with 60 extra frames\).
Playing video.mkv on left of split at 1060 \(with 60 extra frames\).

The second line appears after typing Shift+h to increase the proposed frame index by 60 after the first clip finishes. That means the second clip played will be from frame 1000 to frame 1060, so about two seconds (60 frames) long starting at approximately 33 seconds into the video. You can continue to adjust the position until you've decided you've determined the correct frame.

Once you have found the start and end frames (here 1060 to 2034), you can save the clip to a file:

$ ./encode_frames video.mkv 1060 2034 clip.mkv av

(Change the av to audio if you only want the audio and not the video.)

The details#

The problem#

Last time, I presented a script read_keypress.sh which would read any single keypress in a shell script. For printable characters, it's straightforward what this means, but for keys like left arrow (🠈), it's not obvious how to deal with them.

The solution#

keytest.sh demonstrates using read_keypress.sh to respond to keypresses including 🠈 and Ctrl+c:

#!/bin/bash

esc=$(printf '\e')
ctrlc=$(printf '\x03')

while true
do
  key="$(./read_keypress.sh)"
  case $key in
    q|"$ctrlc")
      echo "Quitting..."
      exit
      ;;
    "${esc}[D")
      echo "Moved left."
      ;;
    *)
      echo "Unexpected key: $key ($(echo -n "$key" | xxd))"
      ;;
  esac
done

The details#

The problem#

For certain applications it can be useful to get many quick responses from the computer, so we would like it to react to individual keypresses instead of requiring an entire command to be typed out and confirmed by pressing Enter. While this is a common feature of GUI and TUI toolkits (e.g., ncurses), it can also be useful for very lightweight inactive experiences written as simple shell scripts.

The solution#

read_keypress.sh will wait for the next keypress and return it as a string:

#!/bin/sh

STTY=$(stty --save)
stty raw -echo
SEL=$(dd if=/dev/tty bs=1k count=1 2>/dev/null)
stty "$STTY"
echo -n "$SEL"

The details#

The problem#

Previously, I wrote a script for opening and immediately focusing xfce4-appfinder. But xfce4-appfinder will notice if it's already open and just assume you want to use the existing window. Even if it's on a different desktop. And therefore attempting to focus it will either do nothing or switch to that desktop, neither of which is desirable.

The solution#

The actual solution I settled on is more of a workaround than a solution: having xfce4-appfinder on a different desktop doesn't really make sense, so I just set it to be on all desktops (specified as the non-existent desktop -1), so it would never be on the wrong desktop:

xdotool set_desktop_for_window "$winid" -1

To actually move the window to the current desktop, replace xdotool windowactivate "$winid" with

desktop="$(xdotool get_desktop)"
win_desktop="$(xdotool get_desktop_for_window "$winid")"
if [[ "$desktop" == "$win_desktop" ]]
then
  xdotool windowactivate "$winid"
else
  xdotool set_desktop_for_window "$winid" -1
  xdotool set_desktop_for_window "$winid" "$desktop" \
          windowactivate "$winid"
fi

Although this assumes that you know the $winid of the window you want to move. If you have just the title it works just as well to use

wmctrl -R "Application Finder"

The details#

The problem#

I have my window manager set to not focus new windows because I dislike having a new window pop up while typing and having the keystrokes surprisingly sent to the new window instead of the one I thought I was typing in. While this is usually what I want, this does mean extra clicks when I did mean to open the new window.

This is particularly bad for xfce4-appfinder (or any other application launcher), since the purpose to be able to set a global keyboard shortcut like Super+Space so you can press that combination and quickly type in the application or action you want (or, even better, type just the first few characters of its name). And since it's being intentionally launched by a keyboard shortcut, there's no real concern of it grabbing keyboard focus unexpectedly.

The solution#

Put the following script in a file appfinder-and-focus.sh and set the keyboard shortcut to run it instead of just running xfce4-appfinder directly:

#!/usr/bin/bash
(xprop -spy -root _NET_CLIENT_LIST | stdbuf -oL tail -n +2 |
  while read -r line
  do
    winid="${line/#*, /}"
    if [[ $(xdotool getwindowname "$winid") == \
        "Application Finder" ]]
    then
      xdotool windowactivate "$winid"
    fi
  done) 2>/dev/null &
xfce4-appfinder &

# Wait for window to appear, then kill xprop.
xdotool search --sync --name "Application Finder" >/dev/null
pkill -P "$(jobs -p %1)"

The details#

The problem#

Last week, I got all of the Factorio saves I had been keeping around into a single directory in order by the time they were created. But what should we do with that data? We could load arbitrary saves to see what our base looked like in the past, but loading the saves individually isn't a great way to do that when there's a lot of them.

The solution#

Luckily, I'm not the only one to want screenshots of my Factorio bases, so there are existing mods to do so.

The FactorioMaps Timelapse mod will take a list of saves and generates a web page like this demo that lets you look around your base across time. As the documentation explains, this is not actually a mod you enable for your save, but a script that you place in your mods/ directory that will run Factorio repeatedly to generate screenshots.

To set it up, use a non-Steam install of Factorio and put the saves you want under its saves directory (in a subdirectory named to_screenshot/ in this example). If you downloaded the mod as ZIP file (e.g., by installing the mod from within Factorio), unzip it; alternatively you can clone the GitHub repo or my fork which includes a few minor improvements for when displaying a lot of saves, especially ones less than an hour apart.

# get to Factorio install /mods/ directory
cd factorio/mods
# clone the git repo with the mod's internal name
git clone https://github.com/dperelman/FactorioMaps.git L0laapk3_FactorioMaps
cd L0laapk3_FactorioMaps
# install dependencies
python -m venv .venv
. .venv/bin/activate
pip install --upgrade -r requirements.txt
# add saves in /saves/to_screenshot/ to timelapse "mybase"
python auto.py --standalone mybase to_screenshot/*

Then you can find the timelapse in the directory script-output/FactorioMaps/mybase/ of your Factorio install; just open index.html in any web browser. Especially if you have a lot of saves, consider adding the --dayonly option to not take twice as much time also generating screenshots of the night view.

Generating screenshots as you play#

Note that if you just want to take screenshots automatically as you play, you can use the Screenshot Toolkit mod, which is what I use for single player games. But due to the way Factorio multiplayer works, doing so impacts every player, so in a multiplayer game it may be better to just copy the saves as the game runs and generate the screenshots later.

The details#

The problem#

Last week, I shared a script which continuously backed up game saves whenever the game saved. The result is a series of directories that contain snapshots of the game saves from every autosave. But to view this data, we really want a list of unique files in order marked with the time they were created.

The solution#

The following will create symbolic links to the unique files named after their modification date:

for i in /tank/factorio/.zfs/snapshot/*/*.zip
do
  ln -sf "$i" "$(stat --printf=%y "$i").zip"
done

or if you want a custom date format, you can use date:

  ln -sf "$i" "$(date -r "$i" +%Y-%m-%d_%H-%M-%S).zip"

Alternatively, the following will just list the unique files with their timestamps:

find /tank/factorio/.zfs/snapshot/ -printf "%T+ %p\n" \
    | sort | uniq --check-chars=30

The details#

The problem#

I was running a Factorio multiplayer server and was being paranoid about making sure I didn't lose any save data. But I also didn't want to put the saves directory on my ZFS file system as it's on a hard drive, not an SSD, and saves taking too long can cause lag for the players (although with non-blocking saving this is much less of an issue).

The solution#

The following script watches the saves/ directory for any new files being written and immediately copies them to the ZFS dataset tank/factorio mounted at /tank/factorio/ and creates a snapshot named with the current date and time. The result is a snapshot corresponding to every time the game saved with the save data.

#!/bin/sh
while true
do
  inotifywait -r saves/ -e close_write
  sleep 0.1s  # write is to *.tmp.zip, wait for rename
  rsync -avhx saves/ /tank/factorio/
  now="$(date +%Y-%m-%d_%H-%M-%S)"
  zfs snapshot tank/factorio@save-"$now"
done

The details#

The problem#

I've recently been writing more series of blog posts or otherwise linking between posts using {filename} links. And also I've been adjusting the scheduling of my future planned blog posts, which involves changing the filename as my naming scheme includes the publication date in the filename. Which means there's opportunities for not adjusting the links to match and ending up with broken links between posts.

Pelican does generate warnings like

WARNING  Unable to find './invalid.md', skipping url        log.py:89
         replacement.                                                

but currently building my entire blog takes about a minute, so I generally only do it when publishing. So I wanted a more lightweight way to just check the intra-blog {filename} links.

The solution#

I wrote the script check_filename_links.sh:

#!/bin/bash

content="${1:-.}"

find "$content" -iname '*.md' -type f -print0 | 
  while IFS= read -r -d '' filename
  do
    grep '^\[.*]: {filename}' "$filename" |
      sed 's/^[^ ]* {filename}\([^\#]*\)\#\?.*$/\1/' |
      while read -r link
      do
        if [ "${link:0:1}" != "/" ]
        then
          linkedfile="$(dirname "$filename")/$link"
        else
          linkedfile="$content$link"
        fi
        if [ ! -f "$linkedfile" ]
        then
          echo "filename=$filename, link=$link,"\
               "file does not exist: $linkedfile"
        fi
      done
  done

Run it from your content/ directory or provide the path to the content/ directory as an argument and it will print out the broken links:

filename=./foo/bar.md, link=./invalid.md, file does not exist: ./foo/./invalid.md

The details#