The problem#

Tametsi (available on Steam¹) is a great logic puzzle game that is a collection of Minesweeper puzzles that can be solved without guessing. The game consists of 100 puzzles plus 60 "bonus" puzzles. The bonus puzzles are in the game directory in an XML-based format that another player has documented well enough that they have even created some puzzles of their own and a viewer for those files. But the base 100 puzzles are nowhere to be found in the puzzle directory, and I had ideas for doing something with them.

The solution#

Given the file DumpPuzzles.java:

public class DumpPuzzles {
    public static void main(String[] args) {
        // Loading puzzles sets the graph on
        //  Game's MouseHandler, so it has to exist.
        game.Game.mh = new io.MouseHandler(null);

        for (int i = 1; i <= 111; i++) {
            puzzle.PuzzleOut.writePuzzle(
                new puzzle.Puzzle(i),
                String.format("puzzle_%03d.puz", i));
        }
    }
}

put it in the same directory as tametsi.exe² and run

$ javac -classpath tametsi.exe DumpPuzzles.java 
$ java -classpath tametsi.exe:. DumpPuzzles

Then the puzzles/ directory will be full of files named puzzle_001.puz, etc.

The details#

The problem#

The Factorio documentation includes a machine-readable version of the prototype API documentation. I wanted to be able to access that information from within a mod, which required somehow giving Lua access to a JSON object.

The solution#

Install my branch of json2lua and run it on your JSON file:

$ npm install git+https://github.com/dperelman/json2lua.git\#feature/string-escaping
$ npx json2lua "FILE.json" "FILE.lua"

Note the output is a Lua table literal, so you'll have to add code to actually assign it to a variable to be able to use it.

The details#

The problem#

Previously, I wrote about using named pipes for IPC to allow controlling a process by another process running on the same computer possibly as a different user, with the access control set by file permissions. But I observed that the restricted unidirectional communication mechanism limited how useful it could be, suggesting another design might be better in settings where bidirectional communication including confirmation of commands may be useful.

Is there a good general solution to this problem without losing the convenience of access control via file permissions?

The solution#

Let's use everyone's favorite RPC mechanism: HTTP. But HTTP normally runs over TCP, and even if we bind to localhost, the HTTP server would still be accessible to any user on the same computer and require selecting a port number that's not already in use. Instead, we can bind the HTTP server to a Unix socket, which similar to named pipes, look a lot like a file, but allow communication like a network socket.

Python's built-in HTTP server doesn't directly support binding to a Unix socket, but the following is slightly modified from an example I found of how to get it to:

import http.server
import json
import os
import socket
import sys
import traceback

def process_cmd(cmd, *args):
    print(f"In process_cmd({cmd}, {args})...")

class HTTPHandler(http.server.BaseHTTPRequestHandler):
    def do_POST(self):
        size = int(self.headers.get('Content-Length', 0))
        body = self.rfile.read(size)
        args = json.loads(body) if body else []
        try:
            result = process_cmd(self.path[1:], *args)
            self.send(200, result or 'Success')
        except Exception:
            self.send(500, str(traceback.format_exc()))

    def do_GET(self):
        self.do_POST()

    def send(self, code, reply):
        # avoid exception in server.py address_string()
        self.client_address = ('',)
        self.send_response(code)
        self.end_headers()
        self.wfile.write(reply.encode('utf-8'))

sock_file = sys.argv[1]
try:
    os.remove(sock_file)
except OSError:
    pass
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
sock.bind(sock_file)
sock.listen(0)

server = http.server.HTTPServer(sock_file, HTTPHandler,
                                False)
server.socket = sock
server.serve_forever()

sock.shutdown(socket.SHUT_RDWR)
sock.close()
os.remove(sock_file)

Then you can query the server using curl:

$ ./server.py http.socket &
$ curl --unix-socket http.socket http://anyhostname/foo
[GET response...]
$ curl --unix-socket http.socket http://anyhostname/foo \
    --json '["some", "args"]'
[POST reponse...]

or in Python, using requests-unixsocket:

import requests_unixsocket
session = requests_unixsocket.Session()
host = "http+unix//http.socket/"
r = session.get(host + "foo")
# Inspect r.status_code and r.text or r.json()
r = session.post(host + "foo", json=["some", "args"])
# Inspect r.status_code and r.text or r.json()

The details#

The problem#

Firefox Marionette only allows a single client to connect a time, so I'd like to have a program in charge of holding that connection that can communicate with the other parts of the system that know what I want Firefox to actually do. While a common way of handling this is to run an HTTP server, that seems pretty heavyweight and would allow access to any user on the machine.

This can be generalized to any case where we want to be able to control one program from another on the same computer. Some reasons why we might not be able to simply act from the first program is if the latter has different access permissions or is holding onto some state like, as mentioned, an open socket.

The solution#

The two programs can communicate over a named pipe, also known as a FIFO. The mkfifo command creates one on the filesystem:

$ mkfifo a_pipe
$ chown server_uid:clients_gid a_pipe
$ chmod 620 a_pipe

Then you can use tail -f a_pipe to watch the pipe and echo something > pipe to write to the pipe. To add a bit more structure, here's a very simple server and client in Python where the client sends one command per line as JSON and the server processes the commands one at a time:

# server.py
import fileinput
import json

def process_cmd(cmd, *args):
    print(f"In process_cmd({cmd})...")

for line in fileinput.input():
    try:
        process_cmd(*json.loads(line))
    except Exception as ex:
        print("Command failed:")
        print(ex, flush=True)

# client.py
import json
import sys

print(json.dumps(sys.argv[1:]))

# Run the server.
$ tail -f a_pipe | python server.py

# Send some commands using the client.
$ python client.py cmd foo bar > a_pipe
$ python client.py another_cmd baz > a_pipe

Then the server will print out

In process_cmd(cmd)...
In process_cmd(another_cmd)...

The details#