The problem#

Schedule grids (example) are a way of displaying a collection of events, some of which occur at overlapping times. They are often used to show what is happening at a conference or other busy event with multiple things going on simultaneously. They are tables where the y-axis is time and the x-axis is sometimes arbitrary or sometimes some concept of location (e.g., which room the event occurs in). Events are rectangles spanning their start through end times and usually covering a single column. This is a fairly standard display format for a calendar application showing a single day's events.

I had a friend who needed schedule grids for their job as a teacher in a (somewhat unusual) classroom that had a lot of small group activities, keeping track of where every student and staff person was supposed to be at all times. They were creating the grids using Google Sheets and spending a lot of time on the layout manually rearranging the columns and manually creating copies of the information to display both a summary and separate schedules for each person.

One complication was that due to privacy laws around information about students, I didn't want any of that data to be touching my server, both because I shouldn't have that data and I don't want to be responsible for promising the school district my server won't get hacked.

The solution#

Schedule Grid Editor (source) is a browser-based tool for creating printable schedule grids. While hosted on my server, the tool works just as well entirely offline. It saves the data in local JSON files (or locally in the browser using OPFS with support for import/export of JSON files).

It maintains a weekly schedule, where each event may be recur on one or more days of the week. Each event has some subset of the students and staff assigned to it, and the logic checks that no one is expected to be at two events simultaneously and that every student is assigned to some event at all times. For each day of the week, it generates a schedule with as few columns as possible showing all of the events. Additionally, for each day of the week, for each person, it generates a single column schedule showing just their events.

The problem#

I had been playing a lot of the social-deduction game The Resistance (and the version with more roles The Resistance: Avalon) and running into the problem that many players had trouble remembering exactly what had happened in previous rounds. Between the fact that there can be several votes throughout a game of the The Resistance and the game can sometimes take up to an hour, it can be hard to remember how the votes went a couple rounds back, especially for teams that were rejected.

The solution#

I created a web app implementation of The Resistance: Avalon (source) using Django.

The problem#

Previously, I figured out how to extract Tametsi's puzzles, but I wanted to make something user-friendly that made use of those puzzles, so I didn't want to require people to install something or run console commands. I had also figured out how to get Java programs running in a browser, so I figured it would be straightforward to combine the two. As you may have guessed from this paragraph being in the "the problem" section, it was not. Specifically, while the Java command-line will read tametsi.exe as a JAR file, Doppio gives the error

Invalid Zip file: Central directory record has invalid signature

The solution#

As a workaround, use a different library to unzip tametsi.exe. There's no need to present it as a JAR file instead of a directory, so no need to rezip it:

var fs = BrowserFS.BFSRequire("fs");

async function unzipToDirectory(zipfile, dir) {
  const z = new zip.fs.FS();
  await z.importBlob(zipfile);

  async function extract(z, dir) {
    fs.mkdir(dir, true)
    if (z.directory) {
      const childDir = z.name
        ? `${dir}/${z.name}`
        : dir;
      for (const child of z.children) {
        await extract(child, childDir);
      }
    } else {
      fs.writeFileSync(
        `${dir}/${z.name}`,
        new buffer(await (await z.getBlob()).arrayBuffer()));
    }
  }

  await extract(z.root, dir);
}

To call that, in the uploadFile() function, replace the reader.onload with

await unzipToDirectory(f, process.cwd() + '/tametsi');

The details#

The problem#

Slashdot recently made a change to their ad code that made the site completely fail to load for me, showing the message

Failed to load website properly since html-load.com is blocked. Please allow html-load.com

and then blaming ad blocking:

This page could not be loaded properly due to incorrect / bad filtering rule(s) of adblockers in use. Please disable all adblockers to continue using the website. (click OK if you'd like to learn more)

I could see the page loaded just fine behind those messages, so it was obviously a lie. But the page continually reloaded if I tried to dismiss those pop-ups, so the site was unusable.

(Actually, this appears to have been disabled in the time it took me to write this blog post, so I guess this is no longer needed. Although it may be applicable to other websites using the same or similar mechanisms.)

The solution#

Install this user script. I have only tested it with Greasemonkey on Firefox so it require modification to work on other user script managers or browsers.

The details#

The problem#

Java and JavaScript are insufficiently easily confused. I had some code I wanted to run on the user's computer and not the server, and it specifically had to be in Java to reference a library written in Java. I found¹ Doppio, a JVM written in TypeScript, but it had bitrotted enough that I was having trouble getting it to run, despite confirming it did what I wanted using the official demo, which provides an in-browser console-like interface.

The solution#

I was able to piece together a working simple page using Doppio, modified from the official example. It's in a repository on GitHub and should be straightforward to modify for your needs.

The details#

The problem#

Last week, I got all of the Factorio saves I had been keeping around into a single directory in order by the time they were created. But what should we do with that data? We could load arbitrary saves to see what our base looked like in the past, but loading the saves individually isn't a great way to do that when there's a lot of them.

The solution#

Luckily, I'm not the only one to want screenshots of my Factorio bases, so there are existing mods to do so.

The FactorioMaps Timelapse mod will take a list of saves and generates a web page like this demo that lets you look around your base across time. As the documentation explains, this is not actually a mod you enable for your save, but a script that you place in your mods/ directory that will run Factorio repeatedly to generate screenshots.

To set it up, use a non-Steam install of Factorio and put the saves you want under its saves directory (in a subdirectory named to_screenshot/ in this example). If you downloaded the mod as ZIP file (e.g., by installing the mod from within Factorio), unzip it; alternatively you can clone the GitHub repo or my fork which includes a few minor improvements for when displaying a lot of saves, especially ones less than an hour apart.

# get to Factorio install /mods/ directory
cd factorio/mods
# clone the git repo with the mod's internal name
git clone https://github.com/dperelman/FactorioMaps.git L0laapk3_FactorioMaps
cd L0laapk3_FactorioMaps
# install dependencies
python -m venv .venv
. .venv/bin/activate
pip install --upgrade -r requirements.txt
# add saves in /saves/to_screenshot/ to timelapse "mybase"
python auto.py --standalone mybase to_screenshot/*

Then you can find the timelapse in the directory script-output/FactorioMaps/mybase/ of your Factorio install; just open index.html in any web browser. Especially if you have a lot of saves, consider adding the --dayonly option to not take twice as much time also generating screenshots of the night view.

Generating screenshots as you play#

Note that if you just want to take screenshots automatically as you play, you can use the Screenshot Toolkit mod, which is what I use for single player games. But due to the way Factorio multiplayer works, doing so impacts every player, so in a multiplayer game it may be better to just copy the saves as the game runs and generate the screenshots later.

The details#

The problem#

Last time, I built a user script that could run on the Folklife 2024 schedule page¹ and reorganize it so it would display the schedule as a grid. But it was brittle and awkward to use because it requiring careful ordering of the interactions with the page and reloading to view a different day's schedule.

The solution#

After failing to come up with an appropriate place to add an event handler, I gave up and took a different approach. I modified the script to work fine if it's run multiple times (and exit quickly if there's no work to do), even if the page is not in a valid state, and then simply set it to rerun every half second. Definitely a hack, but it worked.

Here's the final version of the user script and the Git repo showing the version history.

The details#

The problem#

The Folklife 2024 schedule page¹ is a schedule grid: locations are along the x-axis and time is along the y-axis. Except it's not actually arranged as a grid: each column is just stacked in order with no correspondence to the other columns or the absolute times of the events. Glancing at the code, I noticed the schedule data was available in JSON format, so it should be pretty easy write a user script to display the schedule in a slightly different format.

But when I went to actually make the changes, I found the code is obfuscated React that turned out to be tricky to modify.

The solution#

I was able to write this user script (git repo), which changes the display from the columns of events to a schedule grid. It even works on Firefox mobile, although only if you explicitly request the desktop site.

The details#

The problem#

I was writing a user script where I wanted to be able to intercept the fetch() calls the web page made so my script could use the contents. I found a suggestion of simply reassigning window.fetch to my own function that internally called the real window.fetch while also doing whatever else I wanted. While it worked fine under Tampermonkey on Chromium, under Greasemonkey on Firefox, the script would just silently fail with no indication of why the code wasn't running.

(The script I was writing was this one for fixing the formatting on the Folklife 2024 schedule to reformat the schedule to display as a grid. I plan to write a devlog post on it in the future, but just writing about the most pernicious issue in this post.)

The solution#

The problem was that Firefox's security model special-cases function calls between web pages and extensions (and user scripts running inside Greasemonkey count as part of an extension for this purpose). And, furthermore, Promises can't pass the boundary, so you need to carefully define functions such that the implicit Promise created by declaring the function async lives on the right side of the boundary.

The following code combines all of that together to intercept fetch() on both Firefox and Chromium such that a userscript function intercept is called with the text of the response to every fetch():

const intercept = responseText => {
  // use responseText somehow ...
};

const w = window.wrappedJSObject;
if (w) {
  exportFunction(intercept, window,
                 { defineAs: "extIntercept" });
  w.eval("window.origFetch = window.fetch");

  w.eval(`window.fetch = ${async (...args) => {
    let [resource, config] = args;
    const response = await window.origFetch(resource,config);
    window.extIntercept(await response.clone().text())
    return response;
  }}`);
} else {
  const { fetch: origFetch } = window;

  window.fetch = async (...args) => {
    let [resource, config] = args;
    const response = await origFetch(resource, config);
    intercept(await response.clone().text());
    return response;
  };
}

The details#

The problem#

The straightforward OPFS API for deleting a directory

await parentDir.removeEntry(name, {recursive: true});

doesn't work if the directory contains too many (several hundred) levels of nested directories. The sensible workaround is never create such a directory structure and wipe the OPFS storage entirely if you ever do by accident, but as discussed previously, I did get in that situation due to a bug and wrote some helpers to deal with it.

The solution#

For any reasonable real-world case, what you actually want is probably removeEntry():

await parentDir.removeEntry(name, {recursive: true});

or to delete everything from the root directory:

const root = await navigator.storage.getDirectory();
for await (const handle of root.values()) {
  await root.removeEntry(handle.name, {recursive: true});
}

or possibly to simply use your browser's settings to delete all site data, which should include OPFS data.

In case you do still want to delete a directory in OPFS without worrying about how deeply nested the directory structure is, you can use

async function removeDirectoryFast(dir) {
  const toDelete = [];
  let i = 0;
  let maxDepth = 0;
  for await (const fileHandle
             of getFilesNonRecursively(dir)) {
    maxDepth = Math.max(maxDepth, fileHandle.depth);
    toDelete.push(fileHandle);
  }
  async function deleteAtDepth(depth) {
    for (const f of toDelete) {
      if (f.depth === depth) {
        await f.parentDir.removeEntry(f.name,
                            {recursive: true});
      }
    }
  }
  const increment = 500; // Works empirically in Firefox.
  for (let depth = maxDepth; depth > 1; depth -= increment) {
    await deleteAtDepth(depth);
  }
  await deleteAtDepth(1);
}

This depends on the getFilesNonRecursively() helper from my previous blog post.

The details#