A Weird Imagination

Virtualizing your own computer

Posted in

The problem#

It's annoying to test boot sequences, either of flash drives or your computer. Rebooting every time you want to test a change is slow and loses your place. Which is especially frustrating if you're using a live USB to attempt to repair a broken bootloader and you have to try multiple times. For a bootable flash drive, at least you might have another computer to test it on, making the problem just mildly inconvenient. But what if you didn't have to boot your computer to test the boot sequence?

The solution#

Of course, you have to boot some computer. But it doesn't have to be a physical one: it can be a virtual machine. While virtual machines often use virtual hard disks backed by files, they can also use real disks. And QEMU's -snapshot flag lets you read from a real disk without actually writing back to it. All changes are stored in temporary storage unless you "commit" them, which you do not want to do for this purpose.

IMPORTANT: Always make sure to use -snapshot when running QEMU on a real disk, especially if that disk is in use by the host system. The following commands also have you set the file permissions so QEMU does not have write access to the disks to be extra careful.

The following will run a virtual machine booting off a flash drive (change /dev/sdc to the appropriate device):

# Grant current user read-only access to flash drive
sudo chgrp "$(id -gn)" /dev/sdc
sudo chmod g=r /dev/sdc
# Boot VM off flash drive
qemu-system-x86_64 -snapshot \
    -net none -machine q35 \
    -bios /usr/share/ovmf/OVMF.fd \
    -cpu host -m 8G -enable-kvm \
    /dev/sdc

To instead boot off your machine's internal drive (assuming it is an NVME SSD with device name /dev/nvme0n1):

uefivars -i efivarfs -o edk2 \
    -I /sys/firmware/efi/efivars -O OVMF_VARS.fd
# Grant current user read-only access to primary SSD
sudo chgrp "$(id -gn)" /dev/nvme0n1
sudo chmod g=r /dev/nvme0n1
# Boot VM off primary SSD
qemu-system-x86_64 -snapshot \
    -net none \
    -drive file=/dev/nvme0n1,if=none,id=nvm \
    -device nvme,serial=deadbeef,drive=nvm \
    -machine q35 \
    -drive if=pflash,format=raw,unit=0,readonly=on,\
file=/usr/share/OVMF/OVMF_CODE_4M.fd \
    -drive if=pflash,format=raw,unit=1,file=OVMF_VARS.fd \
    -cpu host -m 8G -enable-kvm

Those commands require QEMU, KVM, and OVMF installed as well as the Python package uefivars. The paths for OVMF are where Debian installed the files on my system, but they may be in a different place or have slightly different filenames on your system.

Note booting the VM may not be instant: that second command takes almost a minute to reach the GRUB menu on my computer.

The details#

Read more…

Installing an OS

Posted in

The problem#

You know how to make a bootable flash drive, but you want to actually use it to install a permanent operating system (OS) onto your computer.

The solution#

Luckily, modern OS installs are very straightforward. Just download the installation image (e.g. Debian Linux or Windows 11), put it on a flash drive, boot off it, and click "Next" a few times and wait a bit. The defaults will usually erase all data on the computer, but if it's a new computer, there's nothing to erase.

Of course, you can make things more complicated if you don't like the defaults, have a special situation, or just want to know more precisely what's going on.

The details#

Read more…

What to boot off flash drives

Posted in

The problem#

Last week, I talked about making bootable flash drives, but didn't go in depth about why you might want to do so.

The solution#

Bootable flash drives have a lot of different uses. The most common ones are simply dealing with an operating system that fails to boot or installing an operating system on a computer that doesn't have one. But there's also some cases where it's useful to not be using the main OS even if it is functional.

The details#

Read more…

Booting off flash drives

Posted in

The problem#

So you've built a new computer with fresh blank storage. How do you actually do anything with that computer that has no software? Navigating the BIOS menus can only hold your interest for so long.

The solution#

The old way of doing things was to have a bootable CD or DVD, but now that most computers don't even have an optical drive, the common way to handle this with bootable USB flash drives.

Most Linux distributions' default download is an image for a bootable "live" flash drive (or DVD) that runs the OS in addition to having an option to perform a permanent install. Some of the most popular ones are Debian, Ubuntu, Mint, and Fedora.

You can boot Windows off a flash drive using Hiren's BootCD, which also includes a lot of recovery and diagnostic tools.

While most boot drives will boot into Linux or Windows, there's a small set of specialized lower-level tools. One very useful one is Memtest86+ (included in many Linux distros), which will determine if your RAM is functional. As bad RAM can cause very weird and different to track down problems, you should always test new RAM.

The details#

Read more…

Testing cheap flash drives

Posted in

The problem#

USB flash drives have gotten very cheap, especially if you don't care too much about the speed or capacity. It's convenient to buy multiple for no more than a few dollars each to always have one available or even to just give away. But sometimes very cheap hardware is non-functional or even counterfeit, claiming to be able to store more data than it really can, so when you try to read that data it will be corrupted or missing.

The solution#

f3 ("Fight Flash Fraud") is a tool for testing flash drives (including SSDs). The basic usage is

( f3write '/mnt/usb/' && f3read '/mnt/usb/' ) | tee f3-log

Replacing /mnt/usb/ with the directory your flash drive is mounted at. If you're testing multiple drives, give the log file a descriptive name to identify which drive the test is for.

(Or use the log-f3wr helper script included with f3 which does basically the same thing.)

That will write files to fill the flash drive and then read them back and verify they contain the same data that was written. This both checks for counterfeit drives as well as failing (or dead-on-arrival) drives. In addition to reporting if the drive is in fact capable of storing as much data as it claims, it will report the average write/read speeds while performing those operations, so it doubles as a simple benchmark.

You should always test new drives before trusting them with real data.

The details#

Read more…

Copy on save

The problem#

I was running a Factorio multiplayer server and was being paranoid about making sure I didn't lose any save data. But I also didn't want to put the saves directory on my ZFS file system as it's on a hard drive, not an SSD, and saves taking too long can cause lag for the players (although with non-blocking saving this is much less of an issue).

The solution#

The following script watches the saves/ directory for any new files being written and immediately copies them to the ZFS dataset tank/factorio mounted at /tank/factorio/ and creates a snapshot named with the current date and time. The result is a snapshot corresponding to every time the game saved with the save data.

#!/bin/sh
while true
do
  inotifywait -r saves/ -e close_write
  sleep 0.1s  # write is to *.tmp.zip, wait for rename
  rsync -avhx saves/ /tank/factorio/
  now="$(date +%Y-%m-%d_%H-%M-%S)"
  zfs snapshot tank/factorio@save-"$now"
done

The details#

Read more…

Resolving apt full-upgrade problems

Posted in

The problem#

My personal desktop runs Debian Unstable ("Sid")1. The nature of running a bleeding edge distro is that things break sometimes. I use Debian Testing/Stable or Ubuntu on my other machines to make my life easier, but I often want access to the latest version of some piece of software and running Debian Unstable is one way to do that. Admittedly, I also do it partially just because fixing things that break is a good way of learning how things work.

The most common kind of problem I run into is that upgrades are not straightforward. For their unstable distro, Debian doesn't make any promises about package dependencies not changing. This is less of a problem when there's an additional package that needs to be installed, but can be complicated when there's conflicts which require removing packages to get an upgrade to go through.

Recently I ran into an extreme version of this problem: trying to upgrade, it proposed uninstalling nearly everything I had installed. Worse, trying to resolve the issue, I got a scary sounding warning that I had uninstalled libssl3:

dpkg: libssl3:amd64: dependency problems, but removing anyway as you requested:
 [...]
 systemd depends on libssl3 (>= 3.0.0).
 sudo depends on libssl3 (>= 3.0.0).
 [...]

Both of those sound important.

The solution#

Luckily, it wasn't as bad as it sounded. Looking at the message, it turned out I had replaced libssl3 with libssl3t64. The latter of which is actually the exact same thing, although the package manager doesn't know that. The reason for the different package name is part of the Debian project to transition to 64-bit time_t, which is required to fix the Year 2038 problem. While on AMD64 and other 64-bit architectures, everything already uses 64-bit time_t, that's not true of all platforms that Debian supports. The way Debian handles ABI transitions like this is to rename the library packages with a suffix (t64 for this one) to ensure the old and new ABI don't get mixed accidentally. Since all of the architectures share the package names, the rename also happens on AMD64 even though there's actual change to match the rename on other platforms where the ABI did change.

Presumably the upgrade will be smoother when done between stable versions, but it really confused apt (which I usually use via wajig):

$ wajig install libssl-dev
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 libegl1 : Depends: libegl-mesa0 but it is not going to be installed
 libreoffice-core : Depends: libgstreamer-plugins-base1.0-0 (>= 1.0.0) but it is not going to be installed
                    Depends: libgstreamer1.0-0 (>= 1.4.0) but it is not going to be installed
                    Depends: liborcus-0.18-0 (>= 0.19.2) but it is not going to be installed
                    Depends: liborcus-parser-0.18-0 (>= 0.19.2) but it is not going to be installed
 wine-development : Depends: wine64-development (>= 8.21~repack-1) but it is not going to be installed or
                             wine32-development (>= 8.21~repack-1)
                    Depends: wine64-development (< 8.21~repack-1.1~) but it is not going to be installed or
                             wine32-development (< 8.21~repack-1.1~)
E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.

Yeah, no idea what libegl1, libreoffice-core, or wine-development have to do with upgrading libssl-dev, but apt was showing those same packages in the error messages no matter what I tried to upgrade and trying to upgrade those packages didn't work either. Luckily, aptitude was able to handle it somewhat better:

$ sudo aptitude install libssl-dev
The following packages will be upgraded:
  libssl-dev{b}
1 packages upgraded, 0 newly installed, 0 to remove and 1459 not upgraded.
Need to get 2,699 kB of archives. After unpacking 1,122 kB will be used.
The following packages have unmet dependencies:
 libssl-dev : Depends: libssl3t64 (= 3.2.1-3) but it is not going to be installed
The following actions will resolve these dependencies:

     Remove the following packages:
1)     libssl3 [3.1.4-2 (now)]
2)     libssl3:i386 [3.1.4-2 (now)]

     Install the following packages:
3)     libssl3t64 [3.2.1-3 (testing, unstable)]
4)     libssl3t64:i386 [3.2.1-3 (testing, unstable)]



Accept this solution? [Y/n/q/?] y
The following NEW packages will be installed:
  libssl3t64{a} libssl3t64:i386{a}
The following packages will be REMOVED:
  libssl3{a} libssl3:i386{a}
The following packages will be upgraded:
  libssl-dev
1 packages upgraded, 2 newly installed, 2 to remove and 1457 not upgraded.
Need to get 7,177 kB of archives. After unpacking 2,294 kB will be used.
Do you want to continue? [Y/n/?]

Getting the packages to upgrade involved a lot of calls to aptitude that looked like that: removing a list of libraries and a installing a matching list of new libraries whose names were identical to those removed except with t64 at the end.

The details#

Read more…

Remote graphical troubleshooting

Posted in

The problem#

For various reasons you might want graphical access to another computer, since some things can't be done over a text interface, including actually designing and troubleshooting what the graphical interface looks like. The other computer might be in a remote location across the internet, in a different room, or simply have a less convenient form factor like a tablet or television, so it's easier to use your desktop's keyboard, mouse, and monitor.

The solution#

The standard solution for this is VNC, specifically the x11vnc VNC server.

To keep a VNC server open to the current X11 session:

x11vnc -usepw -nevershared -forever -localhost -loop &
#... (run one or more graphical applications that block)
# When done, kill everything.
rkill $$

Then to connect to it, assuming the hostname is tablet and you're set up to connect to it via SSH:

$ vncviewer -via tablet -passwd ~/.vnc/tab-passwd localhost

This assumes you've created a ~/.vnc/passwd password file on the server by running

$ x11vnc -storepasswd

and entering something at the prompt from your favorite password generator. No need to save the password anywhere as the file itself is the actual password; just copy it to the client at ~/.vnc/tab-passwd to match the path used in the example above.

The details#

Read more…

Streams and socket and pipes, oh my

You know, like "lions and tigers and bears, oh my"… okay, not funny, moving on…

The problem#

There's a lot of different ways to transmit streams of bytes between applications on the same host or different hosts with various reasons you might want to use each one. And sometimes the two endpoints might disagree on which one they want to be using.

The solution#

As it turns out, there actually is a single answer to bridging any two byte streams: socat. The documentation has plenty of examples. Here's a few I made up involving named pipes and Unix sockets to go along with my recent posts:

Bridge a pair of named pipes to a Unix socket#

socat UNIX-LISTEN:test.sock 'PIPE:pipe_in!!PIPE:pipe_out'

Builds a bridge such that a client sees a Unix socket test.sock and the server communicates through two named pipes, pipe-in to send data over the socket and pipe_out to read the data received over the socket.

Connect to Unix socket HTTP server via TCP#

socat TCP-LISTEN:8042,fork,bind=localhost \
    UNIX-CONNECT:http.sock

For an HTTP server accepting connections via the Unix socket http.sock, makes it also accept connections via the TCP socket localhost:8042.

Forward a Unix socket over an SSH connection#

socat EXEC:"ssh remote 'socat UNIX-CLIENT:service.sock -'" \
    UNIX-LISTEN:proxy-to-remote.sock

Note ssh can do the same without socat (including supporting either side being a TCP port):

ssh -N -L ./proxy-to-remote.sock:./service.sock remote

But that demonstrates combining socat and ssh for getting access to streams only accessible from a remote computer.

The details#

Read more…

Limit processor usage of multiple processes

Posted in

The problem#

In last week's post, I discussed using cpulimit on multiple processes in the special case of web browsers, but I wanted a more general solution.

The solution#

cpulimit-all.sh is a wrapper around cpulimit which will call cpulimit many times to cover multiple processes of the same name and subprocesses.

Using that script, the follow is the equivalent of the script from last week to limit all browser processes to 10% CPU:

cpulimit-all.sh --limit=10 --max-depth=1 \
    -e firefox -e firefox-esr -e chromium -e chrome

But also, we can add a couple options to include any grandchild processes and check for new processes to limit every minute:

cpulimit-all.sh --limit=10 --max-depth=2 \
    -e firefox -e firefox-esr -e chromium -e chrome \
    --watch-interval=1m

The details#

Read more…