You know, like "lions and tigers and bears, oh my"… okay, not funny, moving on…

The problem#

There's a lot of different ways to transmit streams of bytes between applications on the same host or different hosts with various reasons you might want to use each one. And sometimes the two endpoints might disagree on which one they want to be using.

The solution#

As it turns out, there actually is a single answer to bridging any two byte streams: socat. The documentation has plenty of examples. Here's a few I made up involving named pipes and Unix sockets to go along with my recent posts:

Bridge a pair of named pipes to a Unix socket#

socat UNIX-LISTEN:test.sock 'PIPE:pipe_in!!PIPE:pipe_out'

Builds a bridge such that a client sees a Unix socket test.sock and the server communicates through two named pipes, pipe-in to send data over the socket and pipe_out to read the data received over the socket.

Connect to Unix socket HTTP server via TCP#

socat TCP-LISTEN:8042,fork,bind=localhost \
    UNIX-CONNECT:http.sock

For an HTTP server accepting connections via the Unix socket http.sock, makes it also accept connections via the TCP socket localhost:8042.

Forward a Unix socket over an SSH connection#

socat EXEC:"ssh remote 'socat UNIX-CLIENT:service.sock -'" \
    UNIX-LISTEN:proxy-to-remote.sock

Note ssh can do the same without socat (including supporting either side being a TCP port):

ssh -N -L ./proxy-to-remote.sock:./service.sock remote

But that demonstrates combining socat and ssh for getting access to streams only accessible from a remote computer.

The details#

The problem#

In last week's post, I discussed using cpulimit on multiple processes in the special case of web browsers, but I wanted a more general solution.

The solution#

cpulimit-all.sh is a wrapper around cpulimit which will call cpulimit many times to cover multiple processes of the same name and subprocesses.

Using that script, the follow is the equivalent of the script from last week to limit all browser processes to 10% CPU:

cpulimit-all.sh --limit=10 --max-depth=1 \
    -e firefox -e firefox-esr -e chromium -e chrome

But also, we can add a couple options to include any grandchild processes and check for new processes to limit every minute:

cpulimit-all.sh --limit=10 --max-depth=2 \
    -e firefox -e firefox-esr -e chromium -e chrome \
    --watch-interval=1m

The details#

The problem#

cpulimit is a useful utility for stopping a program from wasting CPU, but it only limits a single process. As all modern web browsers use process isolation, limiting just a single process doesn't do very much, we actually want to limit all of the browser processes.

The solution#

The following script will limit the CPU usage of all browser processes to $LIMIT percent CPU. Note that the limit is per process not total over all processes, so you may want to set it quite low to actually have an effect.

LIMIT=10 # Hard-code a limit of 10% CPU as an example.

# Kill child processes (stop limiting CPU) on script exit.
for sig in INT QUIT HUP TERM; do
  trap "
    pkill -P $$
    trap - $sig EXIT
    kill -s $sig "'"$$"' "$sig"
done
trap cleanup EXIT

# Find and limit all child processes of all browsers.
for name in firefox firefox-esr chromium chrome
do
    for ppid in $(pgrep "$name")
    do
        cpulimit --pid="$ppid" --limit="$LIMIT" &
        for pid in "$ppid" $(pgrep --parent "$ppid")
        do
            cpulimit --pid="$pid" --limit="$LIMIT" &
        done
    done
done

The details#

The problem#

My previous post got the video from my smartphone to show up as a camera device on my desktop, but for a video chat, we probably also want audio. So, now the question is: how to build GStreamer pipelines that will allow minimal-webrtc-gstreamer to use virtual microphone and speaker devices that I can point a voice/video chat application at, allowing me to use my smartphone's microphone and speaker for applications on my desktop.

The solution#

The following requires that you are using PulseAudio as your sound server and have downloaded minimal-webrtc-gstreamer:

pactl load-module module-null-sink sink_name=virtspk \
    sink_properties=device.description=Virtual_Speaker
pactl load-module module-null-sink sink_name=virtmic \
    sink_properties=device.description=Virtual_Microphone_Sink
pactl load-module module-remap-source \
    master=virtmic.monitor source_name=virtmic \
    source_properties=device.description=Virtual_Microphone
./minimal-webrtc-host.py\
    --url "https://apps.aweirdimagination.net/camera/"\
    --receiveAudioTo device=virtmic\
    --sendAudio "pulsesrc device=virtspk.monitor"\
    --sendVideo false --receiveVideo false

You can reset your PulseAudio configuration by killing PulseAudio:

pulseaudio -k

You can make the PulseAudio settings permanent by following these instructions to put them in your default.pa file.

The details#

The problem#

I want to make my smartphone's camera appear as an actual camera device on my desktop so any application (primarily Discord) can use it like it were a normal USB web cam.

My previous post introduced minimal-webrtc-gstreamer, which got as far as getting the video stream from any web browser into a GStreamer pipeline, which reduces the problem to outputting a GStreamer pipeline into a virtual web cam device.

The solution#

Download minimal-webrtc-gstreamer and install v4l2loopback. Then run

sudo modprobe v4l2loopback video_nr="42"\
    'card_label=virtcam'\
    exclusive_caps=1 max_buffers=2
./minimal-webrtc-host.py\
    --url "https://apps.aweirdimagination.net/camera/"\
    --receiveVideoTo /dev/video42\
    --sendAudio false

You can test by watching the stream with

gst-launch-1.0 v4l2src device=/dev/video42 ! autovideosink

Note that some applications, including the current desktop release of Discord may not support the virtual camera, showing a solid black square or failing to connect to it at all. It should work in the latest Chromium/Chrome browser, including for the Discord web app.

When done, remove the virtual camera device:

sudo modprobe -r v4l2loopback

The details#

The problem#

When writing a shell script that starts background jobs, sometimes running those jobs past the lifetime of the script doesn't make sense. (Of course, sometimes background jobs really should keeping going after the script completes, but that's not the case this post is concerned with.) In the case that either the background jobs are used to do some background computation relevant to the script or the script can conceptually be thought of as a collection of processes, it makes sense for killing the script to also kill any background jobs it started.

The solution#

At the start of the script, add

cleanup() {
    # kill all processes whose parent is this process
    pkill -P $$
}

for sig in INT QUIT HUP TERM; do
  trap "
    cleanup
    trap - $sig EXIT
    kill -s $sig "'"$$"' "$sig"
done
trap cleanup EXIT

If you really want to kill only jobs and not all child processes, use the kill_child_jobs() function from all.sh or look at the other versions in the kill-child-jobs repository.

The details#

The problem#

My existing script for publishing my blog has Pelican run on the web server and generate the static site directly into the directory served by nginx. This has the effect that while the blog is being published, it is inaccessible or some of the pages or styles are missing. The publish takes well under a minute, so this isn't a big issue, but there's no reason for any downtime at all.

The solution#

Instead of serving the output/ directory, instead generate it and then copy it over by changing the make publish line in schedule_publish.sh to the following:

make publish || exit 1
if [ -L output_dir ]
then
    cp -r output output_dir/
    rm -rf output_dir/html.old
    mv output_dir/html output_dir/html.old
    mv output_dir/output output_dir/html
fi

where output_dir/ is a symbolic link to the parent of the directory actually being served and html/ is the directory actually being served (which output/ previously was a symbolic link to).

The details#

The problem#

Every game, no matter how old and simple, I run on my computer constantly uses an entire CPU thread even when idling at a menu. (Except for some newer multi-threaded games that do the same with multiple threads!) To raise this from a curiosity to a problem, this means that my computer's fans are on at full blast whenever I have a game going, so I notice.

The solution#

To be clear, that symptom could be the result of many different possible causes, others of which I may explore in future blog posts.¹ But specifically for systems with Nvidia GPUs using the Nvidia proprietary driver (as opposed to nouveau), setting the environmental variable __GL_YIELD to USLEEP fixed the issue in some games for me. To do so when running a single game, run __GL_YIELD="USLEEP" /path/to/game or to do so permanently, add the line

export __GL_YIELD="USLEEP"

to ~/.profile and restart X.

The details#

The problem#

The monitor-lock.py script in my previous blog post uses python-xlib, which currently mainly relies on manually porting Xlib functions to Python. This is why it is missing the barrier-related functions I needed in that post. There is work on automating this process, but it appears to be abandoned. I started trying to pick up where they had left off before finding the python-xcffib project which provides auto-generated bindings for libxcb and therefore gives full support for interacting with X at a low level from Python.

python-xcffib (named after the cffi library it uses for binding to the C XCB library) gives a slightly lower-level API than python-xlib, but they are both fairly thin wrappers over the X protocol, so the differences are minor. It was fairly straightforward to port my script from the previous post to use python-xcffib, available as monitor-lock-xcb.py.

Unfortunately, I ran into a bug in python-xcffib:

Traceback (most recent call last):
...
  File "./monitor-lock-xcb.py", line 38, in main
    devices = conn.xinput.XIQueryDevice(xcffib.xinput.Device.AllMaster).reply().infos
...
  File "/usr/lib/python3/dist-packages/xcffib/__init__.py", line 139, in _resize
    assert self.size + increment <= self.known_max
AssertionError

The solution#

I've submitted the fix upstream, so most likely you will not encounter this error. Updating to the latest version (after v0.8.1) should be sufficient to fix the problem.

The fix I applied was to modify the module's __init__.py (the location, which may be different on your machine, is in the stack trace). Specifically, on line 108 in the function Unpacker.unpack(), in the call to struct.calcsize(), change fmt to "=" + fmt.

The details#

The problem#

I hosted a LAN party¹ a little while ago and ended up needing to loan out multiple computers to guests in the interest of having no one try to lug their desktop over. As it turns out, I don't keep multiple of spare gaming-ready laptops around, so I needed to get more computers somehow.

The solution#

My desktop has three screens attached to it (two monitors plus a projector), so given an extra keyboard and mouse (or two), it should be possible to run multiple instances of the game on it at the same time to let multiple people play using the same computer.

The script from this forum post makes it easy to set up multi-pointer X so a second keyboard and mouse will get its own mouse cursor. Then each keyboard and mouse pair can interact with its own instance of the game.

As an additional aid, I wrote monitor-lock.py which allows you to assign a mouse to a monitor, so it cannot be moved off that monitor to prevent accidentally interacting with the other player's instance of the game.

The basic usage is that you first run it with no arguments to get the available screens and pointers getting an output something like this:

$ ./monitor-lock.py 
...
Available screens:
screen 0: {'x': 0, 'y': 0, 'width': 3840, 'height': 2160}
screen 1: {'x': 3840, 'y': 0, 'width': 1920, 'height': 1200}
screen 2: {'x': 3840, 'y': 1200, 'width': 1920, 'height': 1080}

Available pointers:
device 2: Virtual core pointer
device 17: second pointer

USAGE: ./monitor-lock.py [device] [screen]

and then in a screen session (so you don't have to worry about accidentally doing this on a monitor you've locked your pointer away from), run

./monitor-lock.py 2 0

and

./monitor-lock.py 17 1

to lock the primary pointer to the first screen and the second pointer to the second screen.

Just use Ctrl+C to kill the process when you want the pointer to be able to move freely again.

The details#