The problem#

Schedule grids (example) are a way of displaying a collection of events, some of which occur at overlapping times. They are often used to show what is happening at a conference or other busy event with multiple things going on simultaneously. They are tables where the y-axis is time and the x-axis is sometimes arbitrary or sometimes some concept of location (e.g., which room the event occurs in). Events are rectangles spanning their start through end times and usually covering a single column. This is a fairly standard display format for a calendar application showing a single day's events.

I had a friend who needed schedule grids for their job as a teacher in a (somewhat unusual) classroom that had a lot of small group activities, keeping track of where every student and staff person was supposed to be at all times. They were creating the grids using Google Sheets and spending a lot of time on the layout manually rearranging the columns and manually creating copies of the information to display both a summary and separate schedules for each person.

One complication was that due to privacy laws around information about students, I didn't want any of that data to be touching my server, both because I shouldn't have that data and I don't want to be responsible for promising the school district my server won't get hacked.

The solution#

Schedule Grid Editor (source) is a browser-based tool for creating printable schedule grids. While hosted on my server, the tool works just as well entirely offline. It saves the data in local JSON files (or locally in the browser using OPFS with support for import/export of JSON files).

It maintains a weekly schedule, where each event may be recur on one or more days of the week. Each event has some subset of the students and staff assigned to it, and the logic checks that no one is expected to be at two events simultaneously and that every student is assigned to some event at all times. For each day of the week, it generates a schedule with as few columns as possible showing all of the events. Additionally, for each day of the week, for each person, it generates a single column schedule showing just their events.

The problem#

I had been playing a lot of the social-deduction game The Resistance (and the version with more roles The Resistance: Avalon) and running into the problem that many players had trouble remembering exactly what had happened in previous rounds. Between the fact that there can be several votes throughout a game of the The Resistance and the game can sometimes take up to an hour, it can be hard to remember how the votes went a couple rounds back, especially for teams that were rejected.

The solution#

I created a web app implementation of The Resistance: Avalon (source) using Django.

The problem#

The board game Spirit Island has all of players playing more or less simultaneously, especially when acting on different parts of the map, but requires some bookkeeping to be kept among all of the players. For a normal game of at most four players, this isn't difficult, but the game has rules to allow combining multiple copies to a huge game. My friend group planned a 12-player game and we wanted to figure out how to best keep the game organized. (Unfortunately, we planned this game for April 2020 and it did not happen for obvious reasons.)

The solution#

I developed a web app called fear-tracker (source), so called because the main shared information to keep track of is how much fear each player generates in order to keep track of the correct total generated by all players. It supports entering the fear generated per-spirit and the data is synchronized among any number of devices, so there does not have to be an exact correspondence between players and client devices. It also keeps track of what the current phase is as all of the players have to agree on some synchronization points for when new information is revealed by drawing cards.

The details#

The problem#

Sometimes I have a long video that I only want a shorter section of. Maybe it's a TV show that I want to clip a funny scene out of. Or a video of a concert that I want to clip the individual songs out of so I can put them into my music library. But figuring out exactly where to start and end the clip so there's no weird sounds or flashes due to accidentally including the surrounding video takes some care.

The solution#

In order to determine the right clip location, I wanted an easy way to repeatedly watch and listen to the start or end of a proposed clip and get immediate feedback on adjusting its position frame-by-frame.

find_split is a script that implements this workflow. You give it a video, a proposed frame index to split the video on, a preview context length in number of frames, and whether the preview should be the video before (to the left of) or after (to the right of) the split. It plays the preview window ending or starting at the proposed split and waits for the user to press a key indicating how to move the proposed split and then repeats, starting by playing that preview. Once you've confirmed you've determined the desired frame index, press ` (or just Ctrl+c) to exit. The full keyboard controls are described at the link.

$ ./find_split video.mkv 1000 60 left
Playing video.mkv on left of split at 1000 \(with 60 extra frames\).
Playing video.mkv on left of split at 1060 \(with 60 extra frames\).

The second line appears after typing Shift+h to increase the proposed frame index by 60 after the first clip finishes. That means the second clip played will be from frame 1000 to frame 1060, so about two seconds (60 frames) long starting at approximately 33 seconds into the video. You can continue to adjust the position until you've decided you've determined the correct frame.

Once you have found the start and end frames (here 1060 to 2034), you can save the clip to a file:

$ ./encode_frames video.mkv 1060 2034 clip.mkv av

(Change the av to audio if you only want the audio and not the video.)

The details#

The problem#

When writing blog posts, I like to use Markdown's reference-style links which let you avoid writing URLs inline and instead provide a short name and define it elsewhere in the document. I always put them at the end, which results in the bottom of the Markdown file looking like a bibliography for the post. But then there's the extra task of making sure the references at the bottom of the post are consistent with their usage in the blog post; this isn't a huge problem as usually I add a link and immediately use it, looking at the preview to make sure it got used properly. But sometimes I'll start a post by entering a list of links I expect to use, and sometimes I'll miss something checking the preview.

The solution#

lint_refs.py takes any number of Markdown files as arguments and prints the references that are invalid or not used:

#!/usr/bin/env python

import sys
from pathlib import Path
from markdown import markdown, extensions, postprocessors


class ReferenceProxy(dict):
    def __init__(self, *arg, **kw):
        super(ReferenceProxy, self).__init__(*arg, **kw)
        self.read = set()

    def __contains__(self, key):
        self.read.add(key)
        return super().__contains__(key)


class ReferenceLintExtension(extensions.Extension,
                             postprocessors.Postprocessor):
    def __init__(self, filename):
        self.filename = filename

    def extendMarkdown(self, md):
        self.refs = md.references = \
                ReferenceProxy(**md.references)
        md.postprocessors.register(self, 'ref_lint', 1)

    def run(self, text):
        undefined = self.refs.read - set(self.refs.keys())
        unused = set(self.refs.keys()) - self.refs.read
        if unused or undefined:
            print(f"\n\n# {self.filename}")
            if undefined:
                print("\n## UNDEFINED REFERENCES")
                print('\n'.join(sorted(undefined)))
            if unused:
                print("\n## UNUSED REFERENCES")
                print('\n'.join(sorted(unused)))
        return text


for filename in sys.argv[1:]:
    markdown(Path(filename).read_text(), extensions=[
        ReferenceLintExtension(filename),
        'markdown.extensions.extra'])

Given example.md:

A [broken link][broken]. A [working link][working].

[working]: https://example.com/
[not-used]: https://example.org/

$ ./lint-refs.py example.md 


# example.md

## UNDEFINED REFERENCES
broken
broken link

## UNUSED REFERENCES
not-used

The details#

The problem#

Last time, I presented a script read_keypress.sh which would read any single keypress in a shell script. For printable characters, it's straightforward what this means, but for keys like left arrow (🠈), it's not obvious how to deal with them.

The solution#

keytest.sh demonstrates using read_keypress.sh to respond to keypresses including 🠈 and Ctrl+c:

#!/bin/bash

esc=$(printf '\e')
ctrlc=$(printf '\x03')

while true
do
  key="$(./read_keypress.sh)"
  case $key in
    q|"$ctrlc")
      echo "Quitting..."
      exit
      ;;
    "${esc}[D")
      echo "Moved left."
      ;;
    *)
      echo "Unexpected key: $key ($(echo -n "$key" | xxd))"
      ;;
  esac
done

The details#

The problem#

For certain applications it can be useful to get many quick responses from the computer, so we would like it to react to individual keypresses instead of requiring an entire command to be typed out and confirmed by pressing Enter. While this is a common feature of GUI and TUI toolkits (e.g., ncurses), it can also be useful for very lightweight inactive experiences written as simple shell scripts.

The solution#

read_keypress.sh will wait for the next keypress and return it as a string:

#!/bin/sh

STTY=$(stty --save)
stty raw -echo
SEL=$(dd if=/dev/tty bs=1k count=1 2>/dev/null)
stty "$STTY"
echo -n "$SEL"

The details#

The problem#

While I use the command-line and keyboard-based interfaces a lot to manage files, I also regularly use graphical and mouse-based interfaces. As with any time there's different ways of interacting with a system, there's some times when in one mode when you want the features of the other.

The solution#

Xfce's file manager Thunar has a feature called custom actions, which lets you add menu items that run commands on files or directories you are viewing or selecting.

The documentation suggests multiple ideas of actions you might want, including converting among different formants, rotating images, and various file management actions like viewing disk usage and bulk moving files. Additionally, the default action "Open Terminal Here" is one I use frequently.

The details#

The problem#

My Android phone's camera app has an option to take a "motion photo", which, similar to the iOS "live photo" feature, records a short, silent video along with the photo. When viewing the photos on the phone, there's an option to play the video. But image viewer programs on my computer like Eye of GNOME (eog) do not support playing them.

The solution#

ExifTool (in Debian, the package name is libimage-exiftool-perl) can extract the video to a file using this example from a forum post:

exiftool -b -EmbeddedVideoFile photo.jpg > photo_motion.mp4

You can also play it directly without saving it to a file by piping to VLC:

exiftool -b -EmbeddedVideoFile photo.jpg | vlc -

The details#

The problem#

Signal is a privacy-focused instant messaging application which ensures end-to-end encryption on all messages and generally goes out of its way to avoid accidentally revealing your conversations to third parties. As is common with security mechanisms, this necessarily adds some friction some some tasks you actually want to do. In particular, this means viewing your message history outside of the official app is unsupported. Which is especially a problem due to the app being tied to a smartphone that will likely only continue being usable for at most several years. So there's no official way to maintain access to your conversations indefinitely in contrast to IRC and Pidgin logs I have going back decades.

The solution#

The Signal Andriod app supports making encrypted backups (there is no way to get messages out of Signal iOS).

When you enable backups, it gives you a "passphrase" (a sequence of 30 digits). If you have lost this passphrase, there is no way to recover it, but you can disable backups and then re-enable them which will generate a fresh passphrase that will be used for your backups going forward. I recommend storing this passphrase in a password manager like KeePassXC.

To actually use the backup file, you'll need to know where it is. The location is shown on the Signal settings under Chats -> Chat backups -> Backup folder. You can copy it to your computer by plugging the phone in with a USB cable and using an MTP client, which is included in most file managers (I use XFCE's Thunar).

Once you have the backup file and passphrase, you can use signalbackup-tools to convert the backup to something human-readable:

signalbackup-tools [input] [passphrase] \
                   --exporthtml [directory] --allhtmlpages

The output does a good job of trying to look like the Signal interface. See the documentation for more options on controlling the output including various options to filter the output to just some conversations or time ranges.

The details#