Calculating π the hard way#

In honor of Pi Day, I was going to try to write a script that computed π in shell, but given the lack of floating point support, I decided it would be too messy. If you want to see hard to follow code to generate π, I highly recommend the IOCCC entry westley.c from 1998, the majority of which is an ASCII art circle which calculates its own area and radius in order to estimate π. The hint file suggests looking at the output of

$ cc -E westley.c

The 2012 entry, endoh2 is also a pretty amazing π calculator.

Getting π#

Instead, I will just generate π the shell way: using another program.

$ python -c 'import math; print(math.pi)'
3.14159265359

Typographic ligatures#

Typographic ligatures are a feature of a font where multiple characters are combined as one to improve readability. For example, if you look closely at the ffi and fl in the following, you'll notice they look slightly different with ligatures (normal):

office, float

and without ligatures:¹

of‌f‌i‌c‌e, f‌l‌o‌a‌t

You can in fact create a font that declares any sequence of characters to be a ligature to be rendered differently. Because the difference is only at the font rendering level, copy and paste still work as expected. This allows us to choose sequences of characters that really act like single characters in a given programming language and render them as such.

The problem#

After using Vim as my primary editor for a while, I find myself trying to use vi-style keyboard shortcuts in other contexts. Usually resulting in :w in middle of whatever I was writing as saving is a natural thing to do when pausing.

There's a few ways to fix this:

1. Get used to the fact that not everything supports vi-style keyboard shortcuts.
2. Change the keyboard shortcuts of the programs I do use.
3. Use Vim for everything.

The first option is suboptimal because vi-style keyboard shortcuts are very useful. Luckily, in many cases there's ways to get them.

The goal#

Given a series of images, present them in a video with a blend effect between the images. The frames between the input frames should gradually transition between the previous and next image. This, and other transition effects, could likely be implemented using a slideshow generator, but it can also be done quite easily using a shell script.

The script#

The final script I wrote is blend.sh. The following commands fetch and run it:

$ wget https://gist.githubusercontent.com/dperelman/2b4b86233aa13d13c0ab/raw/91c7988e27288af8aeb25ade11d0cab90355702f/blend.sh
$ chmod +x blend.sh
$ ./blend.sh slideshow.mkv first.png second.jpg third.gif
$ mplayer slideshow.mkv

The input images may be in any format. The extension of slideshow.mkv will be used by FFmpeg to guess the desired video format (H.264 in a Matroska Multimedia Container for .mkv).

The problem#

While trying to develop a modification to the Pelican source, I was unexpectedly having my installed version of Pelican get run instead of the local version:

$ which pelican
/usr/local/bin/pelican
$ command -v pelican
/usr/bin/pelican

For some reason, which was pointing to the executable I was expecting bash to run, but the Bash builtin command was telling me that bash was running the installed version instead.

The solution#

Use the hash builtin to clear bash's cache of the location of pelican:

$ hash -d pelican
$ command -v pelican
/usr/local/bin/pelican

Physics#

The totalitarian principle is a concept in physics that states

Everything not forbidden is compulsory.

In order words, any observable outcome not forbidden by a physical law will occur. The many-worlds interpretation of quantum mechanics suggests an even stronger statement that for every such outcome, there is some alternate world in which it does occur.

Programming language design#

While

Everything not forbidden is compulsory.

may sound absurd, its contrapositive

Anything not mandatory is forbidden.

sounds more like a rule you would expect in a computer system, although it may not always be desirable. While it does not fully apply, programming languages and command lines are infamously picky about their inputs:

If you forget a comma in your English essay, your teacher doesn’t hand it back and say, I can’t understand any of this.

In contrast, HTML parsers have historically been quite flexible in what they accept. There's reason to believe this lack of strictness was a strength: less technical users could create their web pages using incorrect HTML that would still work. Somewhat related, browser vendors were also able to add their own extensions and explore what could be added to HTML. On the other hand, the result was large amounts of invalid HTML to the point that HTML5 had to add explicit rules for parsing invalid HTML.

Law and security#

Alternatively, the weaker statement,

Everything which is not forbidden is allowed.

enshrines the principle of law that citizens are free to do whatever they will except when explicitly forbidden by a law.

On the other hand, the same principle applied to computer security is called enumerating badness and is widely held to be a bad idea: put simply, while you as the user of your computer want the freedom to do whatever you want on it, you probably don't want arbitrary code which may have been written by malicious actors to have those same freedoms and it's unfeasible to list all the bad programs as much as you may try.

Instead, many modern systems support the reverse,

Everything which is not allowed is forbidden.

or enumerating goodness, in the form of software repositories and app stores. Although some of these implementations unfortunately go against user freedom.

.NET#

While it seems like we have exhausted the variants of this phrasing, as the title of this post suggests, some software systems follow yet another one:

Some things not allowed are compulsory.

Yes, you read that right.

Using the Windows Azure .NET SDK v2.3, in a web role, the web.config file contained the following automatically generated XML:

  <system.diagnostics>
    <trace>
      <listeners>
        <add type="Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener, Microsoft.WindowsAzure.Diagnostics, Version=2.3.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" name="AzureDiagnostics">
          <filter type="" />
        </add>
      </listeners>
    </trace>
  </system.diagnostics>

I was cleaning up warnings in this project hoping narrow down an unrelated issue and saw this warning on the <filter type="" /> tag:

The 'type' attribute is not allowed.

Naturally, I removed the type="" attribute which didn't seem to be doing anything since the warning said it wasn't even allowed. To my surprise, when I ran the code, it failed to run due to the initialization code throwing an exception with the following message:

Required attribute 'type' not found.

Hence, the type attribute is apparently both not allowed and compulsory.

As the message telling me it is not allowed was merely a warning and not an exception, I put it back and decided not to worry too much about it. Removing the <filter /> tag entirely also seemed to work and eliminate the warning as well.

Pelican's default Makefile includes an option make regenerate which uses Pelican's -r/--autoreload option to regenerate the site whenever a file is modified. Combined with the Firefox extension Auto Reload, this makes it easy to keep an eye on how a blog post will be rendered as you author it and to quickly preview theme changes.

The problem#

With just thirty articles, Pelican already takes several seconds to regenerate the site. For publishing a site, this is plenty fast, but for tweaking formatting in a blog post or theme, this is too slow.

The quick solution#

Pelican has an option, --write-selected, which makes it only write out the files listed. Writing just one file takes about half a second on my computer, even though it still has to do some processing for all of the files in order to determine what to write. To use --write-selected, you have to determine the output filename of the article you are editing:

$ pelican -r content -o output -s pelicanconf.py \
    --relative-urls \
    --write-selected output/draft/in-progress-article.html

The right solution#

Optimally, we wouldn't have to tell Pelican which file to output; instead, it would figure out which files could be affected by a change and regenerate only those files.

CPU and memory#

On Linux, the basic way to monitor load is to use top. The only thing top really has going for it is that it is almost certainly available on any system you will ever use. Luckily, there's a better way: htop. htop supports colors and mouse clicks and lists the available key commands at the bottom of the terminal. It also can be customized to your liking. You can start by putting my htoprc in your ~/.config/htop/ directory:

$ mkdir -p ~/.config/htop/
$ cd ~/.config/htop/
$ wget https://gist.githubusercontent.com/dperelman/1e051f5705685cb41f31/raw/3ab9cf17b166120a805d5f76a71ce82452f553b4/htoprc

Or just explore the options yourself.

Hit F1 (or click Help in the bottom-left) to get an explanation of the colors used in the CPU and memory bars and a guide to keystrokes not listed at the bottom.

In my usage, I find insufficient memory is more often the problem than CPU, so I usually leave htop sorted by the MEM% column.

Other resources#

While CPU and memory are the easiest to monitor resources, they are not the only ones. Linux offers a wide variety of system monitors, depending on what resource you want to monitor and what format you want to view it in. This post focuses on real-time viewing with human-friendly displays but most of these have options or variants that support logging historical data in a more machine-friendly format as well.

The bug#

A couple days ago my Twitter screen scraper stopped working in Liferea. I hadn't changed anything, and the script's output at the command-line still looked okay to my inspection, but Liferea started giving the message

Podcasts are a simple extension to RSS: in addition to text and a link, posts can include a file to be downloaded. In the case of podcasts, this is an audio file. Due to this, while many specialized podcast applications exist, any news aggregator will work, although it might not have the best interface for that use case.

My normal workflow for podcasts is to keep track of them in a news aggregator and explicitly download the files to a local folder. While this isn't overly onerous for a weekly podcast, it is a repetitive task that could be automated. More importantly, downloading an entire backlog of dozens of episodes that way would take a while.