The problem#

Previously, I wrote about using named pipes for IPC to allow controlling a process by another process running on the same computer possibly as a different user, with the access control set by file permissions. But I observed that the restricted unidirectional communication mechanism limited how useful it could be, suggesting another design might be better in settings where bidirectional communication including confirmation of commands may be useful.

Is there a good general solution to this problem without losing the convenience of access control via file permissions?

The solution#

Let's use everyone's favorite RPC mechanism: HTTP. But HTTP normally runs over TCP, and even if we bind to localhost, the HTTP server would still be accessible to any user on the same computer and require selecting a port number that's not already in use. Instead, we can bind the HTTP server to a Unix socket, which similar to named pipes, look a lot like a file, but allow communication like a network socket.

Python's built-in HTTP server doesn't directly support binding to a Unix socket, but the following is slightly modified from an example I found of how to get it to:

import http.server
import json
import os
import socket
import sys
import traceback

def process_cmd(cmd, *args):
    print(f"In process_cmd({cmd}, {args})...")

class HTTPHandler(http.server.BaseHTTPRequestHandler):
    def do_POST(self):
        size = int(self.headers.get('Content-Length', 0))
        body = self.rfile.read(size)
        args = json.loads(body) if body else []
        try:
            result = process_cmd(self.path[1:], *args)
            self.send(200, result or 'Success')
        except Exception:
            self.send(500, str(traceback.format_exc()))

    def do_GET(self):
        self.do_POST()

    def send(self, code, reply):
        # avoid exception in server.py address_string()
        self.client_address = ('',)
        self.send_response(code)
        self.end_headers()
        self.wfile.write(reply.encode('utf-8'))

sock_file = sys.argv[1]
try:
    os.remove(sock_file)
except OSError:
    pass
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
sock.bind(sock_file)
sock.listen(0)

server = http.server.HTTPServer(sock_file, HTTPHandler,
                                False)
server.socket = sock
server.serve_forever()

sock.shutdown(socket.SHUT_RDWR)
sock.close()
os.remove(sock_file)

Then you can query the server using curl:

$ ./server.py http.socket &
$ curl --unix-socket http.socket http://anyhostname/foo
[GET response...]
$ curl --unix-socket http.socket http://anyhostname/foo \
    --json '["some", "args"]'
[POST reponse...]

or in Python, using requests-unixsocket:

import requests_unixsocket
session = requests_unixsocket.Session()
host = "http+unix//http.socket/"
r = session.get(host + "foo")
# Inspect r.status_code and r.text or r.json()
r = session.post(host + "foo", json=["some", "args"])
# Inspect r.status_code and r.text or r.json()

The details#

HTTP server support for Unix sockets#

Many HTTP servers actually support binding to Unix sockets. Even if you aren't worried about access control for connections to localhost, it's more convenient to assign servers named files instead of numbered ports. A common setup is to have one or more application servers listening on Unix sockets with nginx between those servers and the client, using ngx_http_upstream's support for Unix sockets.

I found guides on doing so for Go, Rust, node.js, and, of course, nginx since its listen directive supports Unix sockets. Python web servers like Daphne also support them.

Socket permissions#

The example code deletes the socket and recreates it every time it runs to ensure it doesn't fail to open the socket because it's already open. One complication is that then the socket permissions are reset every time the program is run, so you will want to ensure the permissions on the file it creates are what you want.

According to the man page, the only permission bit that matters for Unix sockets on Linux is the write bit (2), but it warns that other systems may not use the file permissions at all, so if you want to be portable to non-Linux systems, you may want to rely entirely on the access control to the directory you put the socket in.

In Python, you can use os.umask() function (the documentation on the umask command gives more detail) to control the permission bits of files your program creates, including Unix sockets. So os.umask(0o0002) would ensure both the user and group have write permissions or os.umask(0o0000) would ensure everyone has write permissions.

To control the group owner of the created file, you can set the sticky bit of the directory you are putting it in (chmod g+s dir), and then files created in the directory will default to having the same group owner as the directory.

Implementing process_cmd()#

The signature for process_cmd() is intentionally the same as in my previous post, so you can use the same process_cmd() function. The difference is that if it returns a string, then that string will be sent to the client. This means that the implementation of the listtabs command can be changed so instead of writing the tabs object to a file, it can return it to the client:

    elif cmd == 'listtabs':
        tabs = []
        # ... compute tabs
        return json.dumps(tabs)

HTTP request methods#

The example above doesn't actually distinguish between GET and POST HTTP request methods other than not supporting arguments for GET. It doesn't matter that much, but you may want to be more careful with which verb is allowed for which command and possibly add some more expressive query string parsing. The simplest would be to just accept the args JSON after the ? in the request path, although that wouldn't be very idiomatic for how HTTP APIs usually look.

HTTP clients with Unix socket support#

Most HTTP clients including all browsers I checked do not support connecting to Unix sockets. As mentioned above, you can always get nginx to proxy to a normal TCP connection, which might be useful for testing.

From the command line, you can use curl with the --unix-socket option shown above. Note that the host section of the URL can then be anything you want as when connecting to a Unix socket, it is not used to decide which server to connect to. Notably, the similar tool wget does not support Unix sockets. You can also get a low-level connection to a socket using these tools.

In Python, you can use the requests-unixsocket library mentioned above. There's also the lower-level option of HTTPUnixSocketConnection:

import json
from httpunixsocketconnection import HTTPUnixSocketConnection

conn = HTTPUnixSocketConnection(unix_socket='http.socket')

# GET
conn.request("GET", "/foo")
# Always call .getresponse() before making another request.
res = conn.getresponse()
status_code = res.status
response_content = res.read().decode("utf-8")

# POST
conn.request("POST", "/foo", json.dumps(["some", "args"]))
res = conn.getresponse()

Other programming lanaguages' HTTP libraries can likely also be convinced to connect to a Unix socket.