A Weird Imagination

Listing files into a file

Posted in

The problem#

$ ls > file

doesn't do what you expect:

$ touch foo
$ touch bar
$ ls > filelist
$ cat filelist
bar
filelist
foo

You probably didn't expect, or want, filelist to be listed in filelist.

The solution#

$ filelist=$(ls); echo "$filelist" >filelist

Store the output of ls into a variable before writing out to the file, so the file doesn't exist yet when running ls. This can be split into two lines:

$ filelist=$(ls)
$ echo "$filelist" >filelist

but just leaving out the semicolon doesn't work:

$ filelist=INCORRECT
$ filelist=$(ls) echo "$filelist" >filelist
$ cat filelist
INCORRECT

The details#

Why list into a file?#

Usually when I am redirecting ls's output to a file, it's to compare with a directory on another computer:

$ scp remote:/path/to/filelist .
$ ls | diff - filelist
...

A too simple workaround#

The simple and obvious workaround is to just put the file in a different directory. Either

$ ls path/to/the/directory > filelist

or

$ ls > path/to/filelist

would put filelist in a different directory so it would never be included in the list. But just pointing out that what we want to do is probably the wrong thing is not really an answer.

A broken workaround#

$ ls | tee filelist >/dev/null
$ cat filelist
bar
foo

Using tee means that the file isn't created until after tee starts running, while with the redirection, the file is created by sh before it runs ls or tee. ls reads the file list soon after it starts and it runs quickly, so it will often complete before tee starts.

But relying on the scheduler is always a bad idea, and, empirically, it rarely actually works.

The actual fix#

Because we really want ls to finish before there's any chance of the file being created, we just store its result in a variable before writing the file. There is one detail to note:

$ filelist=$(ls)
$ echo $filelist
bar foo
$ echo "$filelist"
bar
foo

You need the quotes around $filelist or the newlines will turn into spaces because the quotes make it get sent to echo as a single argument instead of a list of arguments and echo outputs its arguments separated by spaces.

Similar ps problem#

$ ps x | grep vim
15888 pts/29   S+     0:00 grep --color=auto --exclude-dir=.svn vim
18137 ?        Ssl    0:00 gvim ../test.sh

includes the grep process when you probably didn't want it. There's workarounds that involve carefully crafting regular expressions that don't match themselves:

$ ps x | grep vi[m]
18137 ?        Ssl    0:00 gvim ../test.sh

But that's a messy hack. Instead you can use pgrep. Or pidof. Or ps -C. Why are there three very similar commands to do the same thing? Sorry, I have no idea.

Comments

Have something to add? Post a comment by sending an email to comments@aweirdimagination.net. You may use Markdown for formatting.

There are no comments yet.