Introduction#

Factorio is a sandbox automation and logistics game notable for, among other things, very good support for mods. The developers often go out of their way to support features and fix bugs that only affect mods.

I've only just started to dip my toes in the world of Factorio modding, so I'm definitely no authority on the topic. But this post will be about things that weren't obvious to me starting out.

Resources#

As I said, modding is very well supported, which includes comprehensive documentation and tutorials. One detail I'd call out is that I recommend installing FMTK, which provides IDE tooling for writing Factorio mods, including a VS Code extension (don't worry, there's a Vim mode for VS Code). Also you will probably spend a lot of time looking at the log file. Additionally, you can find lots of examples by looking at the many existing mods; a lot them have links to source code repositories from their mod pages, but even if they don't, you can just download them and unzip them.

If you have a question that can't be answered by those resources or a web search, you can ask for help on the modding forum or the #modding-help channel of the official Factorio Discord.

Other tips#

The problem#

Last time, I built a user script that could run on the Folklife 2024 schedule page¹ and reorganize it so it would display the schedule as a grid. But it was brittle and awkward to use because it requiring careful ordering of the interactions with the page and reloading to view a different day's schedule.

The solution#

After failing to come up with an appropriate place to add an event handler, I gave up and took a different approach. I modified the script to work fine if it's run multiple times (and exit quickly if there's no work to do), even if the page is not in a valid state, and then simply set it to rerun every half second. Definitely a hack, but it worked.

Here's the final version of the user script and the Git repo showing the version history.

The details#

The problem#

The Folklife 2024 schedule page¹ is a schedule grid: locations are along the x-axis and time is along the y-axis. Except it's not actually arranged as a grid: each column is just stacked in order with no correspondence to the other columns or the absolute times of the events. Glancing at the code, I noticed the schedule data was available in JSON format, so it should be pretty easy write a user script to display the schedule in a slightly different format.

But when I went to actually make the changes, I found the code is obfuscated React that turned out to be tricky to modify.

The solution#

I was able to write this user script (git repo), which changes the display from the columns of events to a schedule grid. It even works on Firefox mobile, although only if you explicitly request the desktop site.

The details#

The problem#

I had recently done an apt upgrade that included upgrading ZFS and noticed zpool status showed a weird "(non-allocating)" message, which seemed concerning:

$ zpool status
  pool: tank
 state: ONLINE
config:

    NAME         STATE     READ WRITE CKSUM
    tank         ONLINE       0     0     0
      mirror-0   ONLINE       0     0     0
        ata-***  ONLINE       0     0     0  (non-allocating)
        ata-***  ONLINE       0     0     0  (non-allocating)

errors: No known data errors

The solution#

This forum thread suggested the error may be due to a version mismatch between the ZFS tools and the kernel module. I confirmed there was a mismatch:

$ zpool --version
zfs-2.2.3-2
zfs-kmod-2.1.14-1

The easy way to load the new version of a kernel module after an update is to reboot the computer. But if you don't want to do that, here's the general outline of the commands I ran to unload and reload ZFS (run as root):

# Stop using ZFS
$ zfs umount -a
$ zpool export tank
$ service zfs-zed stop
# Remove modules
$ rmmod zfs
$ rmmod spl
# will show error: rmmod: ERROR: Module spl is in use by: ...
# repeatedly rmmod dependencies until spl is removed.

# Reload ZFS
$ modprobe zfs
$ service zfs-zed start
$ zpool import tank

The details#

The problem#

I use KeePassXC as my password manager in Firefox and while sometimes the connection between Firefox and KeePassXC drops and I have to explicitly click reconnect, it recently stopped working entirely.

The solution#

Install the keepassxc-full package instead of the keepassxc package. If you get the browser extension via the webext-keepassxc-browser package, then your package manager will automatically get the right one.

(This only applies to Debian Sid and Trixie or newer.)

The details#

The problem#

I was writing a user script where I wanted to be able to intercept the fetch() calls the web page made so my script could use the contents. I found a suggestion of simply reassigning window.fetch to my own function that internally called the real window.fetch while also doing whatever else I wanted. While it worked fine under Tampermonkey on Chromium, under Greasemonkey on Firefox, the script would just silently fail with no indication of why the code wasn't running.

(The script I was writing was this one for fixing the formatting on the Folklife 2024 schedule to reformat the schedule to display as a grid. I plan to write a devlog post on it in the future, but just writing about the most pernicious issue in this post.)

The solution#

The problem was that Firefox's security model special-cases function calls between web pages and extensions (and user scripts running inside Greasemonkey count as part of an extension for this purpose). And, furthermore, Promises can't pass the boundary, so you need to carefully define functions such that the implicit Promise created by declaring the function async lives on the right side of the boundary.

The following code combines all of that together to intercept fetch() on both Firefox and Chromium such that a userscript function intercept is called with the text of the response to every fetch():

const intercept = responseText => {
  // use responseText somehow ...
};

const w = window.wrappedJSObject;
if (w) {
  exportFunction(intercept, window,
                 { defineAs: "extIntercept" });
  w.eval("window.origFetch = window.fetch");

  w.eval(`window.fetch = ${async (...args) => {
    let [resource, config] = args;
    const response = await window.origFetch(resource,config);
    window.extIntercept(await response.clone().text())
    return response;
  }}`);
} else {
  const { fetch: origFetch } = window;

  window.fetch = async (...args) => {
    let [resource, config] = args;
    const response = await origFetch(resource, config);
    intercept(await response.clone().text());
    return response;
  };
}

The details#

The problem#

My personal desktop runs Debian Unstable ("Sid")¹. The nature of running a bleeding edge distro is that things break sometimes. I use Debian Testing/Stable or Ubuntu on my other machines to make my life easier, but I often want access to the latest version of some piece of software and running Debian Unstable is one way to do that. Admittedly, I also do it partially just because fixing things that break is a good way of learning how things work.

The most common kind of problem I run into is that upgrades are not straightforward. For their unstable distro, Debian doesn't make any promises about package dependencies not changing. This is less of a problem when there's an additional package that needs to be installed, but can be complicated when there's conflicts which require removing packages to get an upgrade to go through.

Recently I ran into an extreme version of this problem: trying to upgrade, it proposed uninstalling nearly everything I had installed. Worse, trying to resolve the issue, I got a scary sounding warning that I had uninstalled libssl3:

dpkg: libssl3:amd64: dependency problems, but removing anyway as you requested:
 [...]
 systemd depends on libssl3 (>= 3.0.0).
 sudo depends on libssl3 (>= 3.0.0).
 [...]

Both of those sound important.

The solution#

Luckily, it wasn't as bad as it sounded. Looking at the message, it turned out I had replaced libssl3 with libssl3t64. The latter of which is actually the exact same thing, although the package manager doesn't know that. The reason for the different package name is part of the Debian project to transition to 64-bit time_t, which is required to fix the Year 2038 problem. While on AMD64 and other 64-bit architectures, everything already uses 64-bit time_t, that's not true of all platforms that Debian supports. The way Debian handles ABI transitions like this is to rename the library packages with a suffix (t64 for this one) to ensure the old and new ABI don't get mixed accidentally. Since all of the architectures share the package names, the rename also happens on AMD64 even though there's actual change to match the rename on other platforms where the ABI did change.

Presumably the upgrade will be smoother when done between stable versions, but it really confused apt (which I usually use via wajig):

$ wajig install libssl-dev
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 libegl1 : Depends: libegl-mesa0 but it is not going to be installed
 libreoffice-core : Depends: libgstreamer-plugins-base1.0-0 (>= 1.0.0) but it is not going to be installed
                    Depends: libgstreamer1.0-0 (>= 1.4.0) but it is not going to be installed
                    Depends: liborcus-0.18-0 (>= 0.19.2) but it is not going to be installed
                    Depends: liborcus-parser-0.18-0 (>= 0.19.2) but it is not going to be installed
 wine-development : Depends: wine64-development (>= 8.21~repack-1) but it is not going to be installed or
                             wine32-development (>= 8.21~repack-1)
                    Depends: wine64-development (< 8.21~repack-1.1~) but it is not going to be installed or
                             wine32-development (< 8.21~repack-1.1~)
E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.

Yeah, no idea what libegl1, libreoffice-core, or wine-development have to do with upgrading libssl-dev, but apt was showing those same packages in the error messages no matter what I tried to upgrade and trying to upgrade those packages didn't work either. Luckily, aptitude was able to handle it somewhat better:

$ sudo aptitude install libssl-dev
The following packages will be upgraded:
  libssl-dev{b}
1 packages upgraded, 0 newly installed, 0 to remove and 1459 not upgraded.
Need to get 2,699 kB of archives. After unpacking 1,122 kB will be used.
The following packages have unmet dependencies:
 libssl-dev : Depends: libssl3t64 (= 3.2.1-3) but it is not going to be installed
The following actions will resolve these dependencies:

     Remove the following packages:
1)     libssl3 [3.1.4-2 (now)]
2)     libssl3:i386 [3.1.4-2 (now)]

     Install the following packages:
3)     libssl3t64 [3.2.1-3 (testing, unstable)]
4)     libssl3t64:i386 [3.2.1-3 (testing, unstable)]



Accept this solution? [Y/n/q/?] y
The following NEW packages will be installed:
  libssl3t64{a} libssl3t64:i386{a}
The following packages will be REMOVED:
  libssl3{a} libssl3:i386{a}
The following packages will be upgraded:
  libssl-dev
1 packages upgraded, 2 newly installed, 2 to remove and 1457 not upgraded.
Need to get 7,177 kB of archives. After unpacking 2,294 kB will be used.
Do you want to continue? [Y/n/?]

Getting the packages to upgrade involved a lot of calls to aptitude that looked like that: removing a list of libraries and a installing a matching list of new libraries whose names were identical to those removed except with t64 at the end.

The details#

The problem#

I wanted to find a specific Unicode character I had used somewhere in some previous blog post. But by the nature of not knowing exactly what character it was, I wasn't sure how to search for it.

The solution#

Instead, I wrote a script based on this post to simply list all of the characters appearing in any file in a given directory:

cat * | sed 's/./&\n/g' | sort -u

Although the output is small, to further reduce the noise, this version strips out the English letters, numbers, and common symbols:

cat * \
    | tr -d 'a-zA-Z0-9!@#$%^&*()_+=`~,./?;:"[]{}<>|\\'"'-" \
    | sed 's/./&\n/g' | sort -u

The details#

The problem#

For various reasons you might want graphical access to another computer, since some things can't be done over a text interface, including actually designing and troubleshooting what the graphical interface looks like. The other computer might be in a remote location across the internet, in a different room, or simply have a less convenient form factor like a tablet or television, so it's easier to use your desktop's keyboard, mouse, and monitor.

The solution#

The standard solution for this is VNC, specifically the x11vnc VNC server.

To keep a VNC server open to the current X11 session:

x11vnc -usepw -nevershared -forever -localhost -loop &
#... (run one or more graphical applications that block)
# When done, kill everything.
rkill $$

Then to connect to it, assuming the hostname is tablet and you're set up to connect to it via SSH:

$ vncviewer -via tablet -passwd ~/.vnc/tab-passwd localhost

This assumes you've created a ~/.vnc/passwd password file on the server by running

$ x11vnc -storepasswd

and entering something at the prompt from your favorite password generator. No need to save the password anywhere as the file itself is the actual password; just copy it to the client at ~/.vnc/tab-passwd to match the path used in the example above.

The details#

You know, like "lions and tigers and bears, oh my"… okay, not funny, moving on…

The problem#

There's a lot of different ways to transmit streams of bytes between applications on the same host or different hosts with various reasons you might want to use each one. And sometimes the two endpoints might disagree on which one they want to be using.

The solution#

As it turns out, there actually is a single answer to bridging any two byte streams: socat. The documentation has plenty of examples. Here's a few I made up involving named pipes and Unix sockets to go along with my recent posts:

Bridge a pair of named pipes to a Unix socket#

socat UNIX-LISTEN:test.sock 'PIPE:pipe_in!!PIPE:pipe_out'

Builds a bridge such that a client sees a Unix socket test.sock and the server communicates through two named pipes, pipe-in to send data over the socket and pipe_out to read the data received over the socket.

Connect to Unix socket HTTP server via TCP#

socat TCP-LISTEN:8042,fork,bind=localhost \
    UNIX-CONNECT:http.sock

For an HTTP server accepting connections via the Unix socket http.sock, makes it also accept connections via the TCP socket localhost:8042.

Forward a Unix socket over an SSH connection#

socat EXEC:"ssh remote 'socat UNIX-CLIENT:service.sock -'" \
    UNIX-LISTEN:proxy-to-remote.sock

Note ssh can do the same without socat (including supporting either side being a TCP port):

ssh -N -L ./proxy-to-remote.sock:./service.sock remote

But that demonstrates combining socat and ssh for getting access to streams only accessible from a remote computer.

The details#