The problem#

When writing a blog post like last time's, I often will be at least partway through the process before I realize it's interesting enough to write a post on. Then I need to somehow go back and reconstruct a narrative of the troubleshooting steps I performed.

The solution#

As long as I still have the terminals or screen sessions open, I can at least capture a snapshot of the scrollback history that's still available.

In Xfce Terminal (and likely similar in other terminals), I can save the history by going into the "Edit" menu and selecting "Select All" and then "Copy" (plain text) or "Copy as HTML" (to also capture styles) and pasting into a file or writing the clipboard to a file using xclip:

xclip -o -selection clipboard > some_file.html

In screen, the hardcopy command can be used to dump the full history to a file. Type Ctrl+a, : to get into command mode and run the command hardcopy -h to include the entire scrollback buffer. Unfortunately, this is plain text only.

If you use tmux instead of screen, the equivalent is the capture-pane command which does support saving styles if you provide the -e option. Note this encodes the styles so cating the file to a terminal will look right, not in HTML as Xfce Terminal does; you could convert it to HTML using aha.

The details#

Scrollback limits#

Normally terminals limit how far you can scroll back before they truncate the history. You can easily observe this by running yes (or seq 9999999 so the lines are numbered), stopping it with Ctrl+c, and scrolling up as far your terminal will let you, and observing that you only see part of the command's output, not the line running the command.

Exactly how many lines get stored is usually configurable. For Xfce Terminal, it's on the "General" preferences tab. For screen, you can set a defscrollback value in ~/.screenrc. For tmux, you can set history-limit in ~/.tmux.conf.

Unlimited scrollback#

Some terminals (Xfce Terminal, Gnome Terminal, and Konsole included) actually support unlimited scrollback. Of course, at some point your computer will run out of storage space, but, when enabled, they write the scrollback to a file instead of storing it in memory and don't set an explicit limit on its size.

Unlimited scrollback file security#

Xfce Terminal and Gnome Terminal are both backed by VTE which encrypts the scrollback file using AES-256-GCM. On the other hand, this post claims Konsole does not encrypt its unlimited scrollback file and warns about this in its configuration. But the version of Konsole on my computer does not have such a warning that I have seen. I did find some discussion of encrypting the file (rejected with the reasoning that the filesystem should be in charge of encryption, not the application) and a request for a limited in-memory-only version, but neither of those requests appear to have been fulfilled. The Konsole configuration does have a setting for where to put the file, so you can select a directory that satisfies your security requirements (e.g., encrypted or tmpfs).

Recording a log#

Once you realize you do want to be saving your terminal history, there's various tools for doing so.

Within a screen session, you can use log command (or the keyboard shortcut Ctrl+a, Shift+H) to log everything printed in the current window to a file screenlog.0 in the directory you ran screen from. As that logs what is actually written to the terminal, that does include the ANSI codes for colors. tmux has similar features.

In any terminal, even if you aren't using screen or tmux, you can record a log using script (or more modern replacements like ttyrec or termrec). While not relevant for my usecase here, these have the additional advantage of supporting timing information so they can replay a log like a video, which is apparently mainly used for NetHack replays.