A Weird Imagination

Pi in shell

Posted in

Calculating π the hard way#

In honor of Pi Day, I was going to try to write a script that computed π in shell, but given the lack of floating point support, I decided it would be too messy. If you want to see hard to follow code to generate π, I highly recommend the IOCCC entry westley.c from 1998, the majority of which is an ASCII art circle which calculates its own area and radius in order to estimate π. The hint file suggests looking at the output of

$ cc -E westley.c

The 2012 entry, endoh2 is also a pretty amazing π calculator.

Getting π#

Instead, I will just generate π the shell way: using another program.

$ python -c 'import math; print(math.pi)'
3.14159265359

Read more…

Making :w work everywhere

Posted in

The problem#

After using Vim as my primary editor for a while, I find myself trying to use vi-style keyboard shortcuts in other contexts. Usually resulting in :w in middle of whatever I was writing as saving is a natural thing to do when pausing.

There's a few ways to fix this:

  1. Get used to the fact that not everything supports vi-style keyboard shortcuts.
  2. Change the keyboard shortcuts of the programs I do use.
  3. Use Vim for everything.

The first option is suboptimal because vi-style keyboard shortcuts are very useful. Luckily, in many cases there's ways to get them.

Read more…

Blend effect slideshow using shell

Posted in

The goal#

Given a series of images, present them in a video with a blend effect between the images. The frames between the input frames should gradually transition between the previous and next image. This, and other transition effects, could likely be implemented using a slideshow generator, but it can also be done quite easily using a shell script.

The script#

The final script I wrote is blend.sh. The following commands fetch and run it:

$ wget https://gist.githubusercontent.com/dperelman/2b4b86233aa13d13c0ab/raw/91c7988e27288af8aeb25ade11d0cab90355702f/blend.sh
$ chmod +x blend.sh
$ ./blend.sh slideshow.mkv first.png second.jpg third.gif
$ mplayer slideshow.mkv

The input images may be in any format. The extension of slideshow.mkv will be used by FFmpeg to guess the desired video format (H.264 in a Matroska Multimedia Container for .mkv).

Read more…

Which command will be run?

Posted in

The problem#

While trying to develop a modification to the Pelican source, I was unexpectedly having my installed version of Pelican get run instead of the local version:

$ which pelican
/usr/local/bin/pelican
$ command -v pelican
/usr/bin/pelican

For some reason, which was pointing to the executable I was expecting bash to run, but the Bash builtin command was telling me that bash was running the installed version instead.

The solution#

Use the hash builtin to clear bash's cache of the location of pelican:

$ hash -d pelican
$ command -v pelican
/usr/local/bin/pelican

Read more…

Monitor all the things

CPU and memory#

On Linux, the basic way to monitor load is to use top. The only thing top really has going for it is that it is almost certainly available on any system you will ever use. Luckily, there's a better way: htop. htop supports colors and mouse clicks and lists the available key commands at the bottom of the terminal. It also can be customized to your liking. You can start by putting my htoprc in your ~/.config/htop/ directory:

$ mkdir -p ~/.config/htop/
$ cd ~/.config/htop/
$ wget https://gist.githubusercontent.com/dperelman/1e051f5705685cb41f31/raw/3ab9cf17b166120a805d5f76a71ce82452f553b4/htoprc

Or just explore the options yourself.

Hit F1 (or click Help in the bottom-left) to get an explanation of the colors used in the CPU and memory bars and a guide to keystrokes not listed at the bottom.

In my usage, I find insufficient memory is more often the problem than CPU, so I usually leave htop sorted by the MEM% column.

Other resources#

While CPU and memory are the easiest to monitor resources, they are not the only ones. Linux offers a wide variety of system monitors, depending on what resource you want to monitor and what format you want to view it in. This post focuses on real-time viewing with human-friendly displays but most of these have options or variants that support logging historical data in a more machine-friendly format as well.

Read more…

Mysterious Twitter scraping bug

Posted in

The bug#

A couple days ago my Twitter screen scraper stopped working in Liferea. I hadn't changed anything, and the script's output at the command-line still looked okay to my inspection, but Liferea started giving the message

Read more…

Download all items in a podcast

Posted in

Podcasts are a simple extension to RSS: in addition to text and a link, posts can include a file to be downloaded. In the case of podcasts, this is an audio file. Due to this, while many specialized podcast applications exist, any news aggregator will work, although it might not have the best interface for that use case.

My normal workflow for podcasts is to keep track of them in a news aggregator and explicitly download the files to a local folder. While this isn't overly onerous for a weekly podcast, it is a repetitive task that could be automated. More importantly, downloading an entire backlog of dozens of episodes that way would take a while.

Read more…

Reverse sequence for tr

The problem#

If you take the word wizard, reverse the order of the letters and reverse the alphabet:

From: abcdefghijklmnopqrstuvwxyz
To:   ZYXWVUTSRQPONMLKJIHGFEDCBA

then you get the word wizard back, an observation made at least as early as 1972.

Now let's write a shell script to verify this so we can find other words with similar interesting properties. The obvious shell script to verify this

echo wizard | tr a-z z-a | rev

unfortunately fails with the error

tr: range-endpoints of 'z-a' are in reverse collating sequence order

The error is by design: it's not clear what a sequence in reverse order should mean, so POSIX actually requires that it not work.

Read more…

Better hash-based colors

The problem#

Yesterday, I proposed using a hash function to choose colors:

ps1_color="32;38;5;$((0x$(hostname | md5sum | cut -f1 -d' ' | tr -d '\n' | tail -c2)))"

I did not discuss two issues with this approach:

  1. Colors may be too similar to other colors, such that they are not useful.
  2. Some colors may be undesirable altogether, particularly very dark ones may be too similar to a black terminal background color.

Read more…

Hash-based hostname colors

Random color selection#

In my post about hostname-based prompt colors, I suggested a fallback color scheme that was obviously wrong in order to remind you to set a color for that host:

alice@unknown:~$ 

This carried with it an implicit assumption: you care what color each host is assigned. You may instead be happy to assign a random color to each host. We could use shuf to generate a random color:

ps1_color="32;38;5;$(shuf -i 0-255 -n 1)"

The problem with this solution is the goal of the recoloring the prompt was not simply to make it more colorful, but for that color to have meaning. We want the color to always be the same for each login to a given host.

One way to accomplish this would be to use that code to randomly generate colors, but save the results in a table like the one used before for manually-chosen colors. But it turns out we can do better.

Hash-based color selection#

Hash functions have a useful property called determinism, which means that hashing the same value will always get the same result. The consequence is that we can use a hash function like it's a lookup table of random numbers shared among all of our computers:

ps1_color="32;38;5;$(($(hostname | sum | cut -f1 -d' ' | sed s/^0*//) % 256))"

The $((...)) syntax is bash's replacement for expr which is less portable but easier to use. Here we use it to make sure the hash value we compute is a number between 0 and 255. [sum][sum] computes a hash of its input, in this case the result of hostname. Its output is not just a number so cut selects out the number and sed gets rid of any leading zeros so it isn't misinterpreted as octal.

The idea of using sum was suggested by a friend after reading my previous post on the topic.

But this turns out to not work great for hosts with similar names like rob.example.com and orb.example.com:

alice@rob:~$ 
alice@orb:~$ 

Similar colors on hosts with very different names would not be so bad, but because of how sum works, it will tend to give similar results on similar strings (although less often than I expected; it took some effort to find such an example).

Better hash functions#

While this is not a security-critical application, here cryptographic hash functions solve the problem. Cryptographic hash functions guarantee (in theory) that knowing that two inputs are similar tells you nothing about their hash values. In other words, the output of cryptographic hash functions are indistinguishable from random and, in fact, they can be used to build pseudorandom generators like Linux's /dev/urandom.

The cryptographic hash function utilities output hex instead of decimal, so they aren't quite a drop-in replacement for sum:

ps1_color="32;38;5;$((0x$(hostname | md5sum | cut -f1 -d' ' | tr -d '\n' | tail -c2)))"

Here we use cut and tr to select just the hex string of the hash. tail's -c option specifies the number of bytes to read from the end, where 2 bytes corresponds to 2 hex digits, which can have a value of 0 to 255, so the modulo operation is not needed. Instead the 0x prefix inside $((...)) interprets the string as a hex number and outputs it as a decimal number.

This code uses the md5sum utility to compute an MD5 hash of the hostname. This is recommended because md5sum is likely to be available on all hosts. Do be aware that MD5 is insecure and it is only okay to use here because coloring the prompt is not a security-critical application.

sha1sum and sha256sum are also likely available on modern systems and work as drop-in replacements for md5sum in the above command should you wish to use a different hash. Additionally, you could also get different values out of the hash by adding a salt:

salt="Some string."
ps1_color="32;38;5;$((0x$( (echo "$salt"; hostname) | sha256sum | cut -f1 -d' ' | tr -d '\n' | tail -c2)))"