The problem#

It's annoying to test boot sequences, either of flash drives or your computer. Rebooting every time you want to test a change is slow and loses your place. Which is especially frustrating if you're using a live USB to attempt to repair a broken bootloader and you have to try multiple times. For a bootable flash drive, at least you might have another computer to test it on, making the problem just mildly inconvenient. But what if you didn't have to boot your computer to test the boot sequence?

The solution#

Of course, you have to boot some computer. But it doesn't have to be a physical one: it can be a virtual machine. While virtual machines often use virtual hard disks backed by files, they can also use real disks. And QEMU's -snapshot flag lets you read from a real disk without actually writing back to it. All changes are stored in temporary storage unless you "commit" them, which you do not want to do for this purpose.

IMPORTANT: Always make sure to use -snapshot when running QEMU on a real disk, especially if that disk is in use by the host system. The following commands also have you set the file permissions so QEMU does not have write access to the disks to be extra careful.

The following will run a virtual machine booting off a flash drive (change /dev/sdc to the appropriate device):

# Grant current user read-only access to flash drive
sudo chgrp "$(id -gn)" /dev/sdc
sudo chmod g=r /dev/sdc
# Boot VM off flash drive
qemu-system-x86_64 -snapshot \
    -net none -machine q35 \
    -bios /usr/share/ovmf/OVMF.fd \
    -cpu host -m 8G -enable-kvm \
    /dev/sdc

To instead boot off your machine's internal drive (assuming it is an NVME SSD with device name /dev/nvme0n1):

uefivars -i efivarfs -o edk2 \
    -I /sys/firmware/efi/efivars -O OVMF_VARS.fd
# Grant current user read-only access to primary SSD
sudo chgrp "$(id -gn)" /dev/nvme0n1
sudo chmod g=r /dev/nvme0n1
# Boot VM off primary SSD
qemu-system-x86_64 -snapshot \
    -net none \
    -drive file=/dev/nvme0n1,if=none,id=nvm \
    -device nvme,serial=deadbeef,drive=nvm \
    -machine q35 \
    -drive if=pflash,format=raw,unit=0,readonly=on,\
file=/usr/share/OVMF/OVMF_CODE_4M.fd \
    -drive if=pflash,format=raw,unit=1,file=OVMF_VARS.fd \
    -cpu host -m 8G -enable-kvm

Those commands require QEMU, KVM, and OVMF installed as well as the Python package uefivars. The paths for OVMF are where Debian installed the files on my system, but they may be in a different place or have slightly different filenames on your system.

Note booting the VM may not be instant: that second command takes almost a minute to reach the GRUB menu on my computer.

The details#

The problem#

You know how to make a bootable flash drive, but you want to actually use it to install a permanent operating system (OS) onto your computer.

The solution#

Luckily, modern OS installs are very straightforward. Just download the installation image (e.g. Debian Linux or Windows 11), put it on a flash drive, boot off it, and click "Next" a few times and wait a bit. The defaults will usually erase all data on the computer, but if it's a new computer, there's nothing to erase.

Of course, you can make things more complicated if you don't like the defaults, have a special situation, or just want to know more precisely what's going on.

The details#

The problem#

Last week, I talked about making bootable flash drives, but didn't go in depth about why you might want to do so.

The solution#

Bootable flash drives have a lot of different uses. The most common ones are simply dealing with an operating system that fails to boot or installing an operating system on a computer that doesn't have one. But there's also some cases where it's useful to not be using the main OS even if it is functional.

The details#

The problem#

So you've built a new computer with fresh blank storage. How do you actually do anything with that computer that has no software? Navigating the BIOS menus can only hold your interest for so long.

The solution#

The old way of doing things was to have a bootable CD or DVD, but now that most computers don't even have an optical drive, the common way to handle this with bootable USB flash drives.

Most Linux distributions' default download is an image for a bootable "live" flash drive (or DVD) that runs the OS in addition to having an option to perform a permanent install. Some of the most popular ones are Debian, Ubuntu, Mint, and Fedora.

You can boot Windows off a flash drive using Hiren's BootCD, which also includes a lot of recovery and diagnostic tools.

While most boot drives will boot into Linux or Windows, there's a small set of specialized lower-level tools. One very useful one is Memtest86+ (included in many Linux distros), which will determine if your RAM is functional. As bad RAM can cause very weird and different to track down problems, you should always test new RAM.

The details#

The problem#

USB flash drives have gotten very cheap, especially if you don't care too much about the speed or capacity. It's convenient to buy multiple for no more than a few dollars each to always have one available or even to just give away. But sometimes very cheap hardware is non-functional or even counterfeit, claiming to be able to store more data than it really can, so when you try to read that data it will be corrupted or missing.

The solution#

f3 ("Fight Flash Fraud") is a tool for testing flash drives (including SSDs). The basic usage is

( f3write '/mnt/usb/' && f3read '/mnt/usb/' ) | tee f3-log

Replacing /mnt/usb/ with the directory your flash drive is mounted at. If you're testing multiple drives, give the log file a descriptive name to identify which drive the test is for.

(Or use the log-f3wr helper script included with f3 which does basically the same thing.)

That will write files to fill the flash drive and then read them back and verify they contain the same data that was written. This both checks for counterfeit drives as well as failing (or dead-on-arrival) drives. In addition to reporting if the drive is in fact capable of storing as much data as it claims, it will report the average write/read speeds while performing those operations, so it doubles as a simple benchmark.

You should always test new drives before trusting them with real data.

The details#

The problem#

Last week, I shared some thoughts on how to buy parts for a homebuilt computer. But what do you do once all the boxes show up?

The solution#

Text is a poor medium for explaining how to physically put things together. I'll share some tips, but for the actual processes, look at the manuals for the components you bought, some of which will probably point to official videos showing how to do things. And there's probably plenty of videos online of people demonstrating every step.

The details#

The problem#

I recently helped a friend with building a computer. While there's plenty of guides online, I thought I'd write down my own thoughts of everything I was considering for this recent computer build.

The intended use of the computer was as an inexpensive but expandable gaming PC. Meaning it should be able to comfortably play relatively recent games at an acceptable framerate, but not at max settings. And hopefully that should continue to be true for at least a year or two. And when it does get to be too underpowered for even low-end gaming, it should be possible to fix that by replacing parts, not the entire computer.

The solution#

Most of planning a computer build is using PCPartPicker and NewEgg to search for the components and NewEgg and Tom's Hardware for reviews (the former for specific components, the latter for selecting things like how powerful a CPU/GPU you need).

I am intentionally avoiding specific component recommendations in this post because those would be out-of-date pretty much immediately.

The PCPartPicker interface will guide you to what categories of components you will need, and help you determine which parts are compatible with each other. The required components for a functioning computer are a motherboard, a CPU (and cooler if not included), memory (RAM), and a power supply. You will almost always also want storage (an SSD), a case, and additional cooling (usually fans to install in the case). Most of the time you will also want a video card (GPU), but modern processors often have a very underpowered one built-in, which is usually sufficient if you aren't playing games or otherwise using a GPU.

Note that I am not including peripherals. To actually use the computer, you will probably want a keyboard, mouse, monitor, and possibly speakers. But those are easily moved among different computers.

The details#

The problem#

Last week, I talked about the modifications I and others wanted to make to the Supply Challenge scenario that comes with Factorio. This week I'll talk about actually making those modifications.

The solution#

If you just want to play the modified version, you can install my mod Supply Challenge Plus (source code on GitHub). The Mod Portal page describes what changes it makes and has screenshots of the UI with various choices for the settings to customize it.

The details#

The problem#

Factorio comes with a scenario called Supply Challenge which is a shorter, more directed experience than the standard "Freeplay" game mode. It replaces the pressure from enemies attacking your base with a series of timed requests where you have to provided a pre-defined set of items within a time limit with a new request every several minutes. This both can be good for new players to have guidance on what they should be working on next and for experienced players as getting everything done within the time limit can be, as the name suggests, a challenge. As those are two somewhat opposing goals, I wanted to add settings to make it better for both use cases.

But first was the question that precedes many coding projects: has someone already done this?¹ And the related question: has anyone suggested doing it and what features did they find important that might be worth considering in the design?

Why?#

For a personal project that possibly no one else is going to use, it's not immediately obvious why I care what features other people might want. But there's a few reasons such a search can be valuable in addition to the obvious that other people might use what I create. First, finding other users wanting the same features I want is validation that those features are good ideas. Other people may have thought of features that I hadn't thought to implement but actually want. And even for features that I am not interested in implementing at the moment, keeping them in mind may affect the design.

Initial ideas#

The problem#

Using my Factorio reflection library discussed previously involves interacting with Lua values that are a combination of the actual value and some metadata, so you have to know about those values to use them. Worse, the interactions I defined are quite verbose. The main thing you're like to want to do on a value is lookup a property on it. Normally in Lua that looks like

table[key]

but if instead of table you have a wrapped value from the reflection library, you would look up key on it with

ReflectionLibraryMod.typed_object_lookup_property(
    wrapped, key).value

If you want to do multiple levels of property lookups, then this quickly gets quite unwieldy.

The solution#

Lua supports operator overloading through a mechanism it calls metatables (some additional examples).

Using that mechanism, the library defines a value ReflectionLibraryMod.typed_data_raw that can be indexed as wrapped[key] and assigned to like wrapped[key] = newValue.

The basic setup looks like

local prototype = {} -- table for methods
local mt = {}
mt.__index = function (table, key)
  local res = prototype[key]
    or ReflectionLibraryMod.wrap_typed_object(
      ReflectionLibraryMod.typed_object_lookup_property(
        table._private, key))
  if res == nil then
    if key == "_value" then
      res = table._private.value
    end
  end
  return res
end

mt.__newindex = function (table, key, newValue)
  -- If newValue is a wrapped typed value, then unwrap it.
  if getmetatable(newValue) == mt then
    newValue = newValue._private.value
  end
  table._private.value[key] = newValue
end

function ReflectionLibraryMod.wrap_typed_object(typedValue)
  if typedValue == nil then
    return nil
  end

  local res = {_private = typedValue}
  setmetatable(res, mt)

  return res
end

Any additional properties would be defined next to the definition of _value. And any methods would be defined on prototype.

The details#