When creating the outline for my book (now officially published and in print!), I decided to organize it around the nine facets of an awesome command-line app. Each chapter focuses on one of these facets. They state that an awesome command-line app should:
- have a clear and concise purpose
- be easy to use
- be helpful
- play well with others
- delight casual users
- make configuration easy for advanced users
- install and distribute painlessly
- be well-tested and as bug free as possible
- be easy to maintain
In this post, I’ll illustrate each of these facets (along with a test of the tenth chapter on color and formatting), via a code walkthrough of a simple command-line app I created for work.
LivingSocial (where I work) processes thousands of credit card transactions per day, across a highly distributed, asynchronous system. When things go wrong, the log files are the first place I look to find answers. This means that
grep is my go-to tool for analysis. Even though
grep can highlight search terms in output, with long and complex log lines, it can be hard to pick out just what I’m looking for. I needed a tool to just highlight text, but not actually “grep out” non-matching lines.
To the command-line!
So, in just a few short hours, hl was born. I wrote it using TDD, and, even though it’s barely 100 lines of code, it hits all the notes of an awesome command-line app (if I do say so myself :). Let’s go through all nine of our “facets of an awesome command-line app” and see what the fuss is about.
Have a Clear & Concise Purpose
The best way to have a clear & concise purpose is to do one thing, and one thing only.
hl highlights search terms in any output to assist with visual scanning of output. It doesn’t highlight multiple terms, and it doesn’t remove non-matching lines. It just highlights terms. One thing, and one thing only.
Be Easy to Use
This is a big topic, but here’s an example of using
hl does what it’s asked, by default, without a lot of fuss, just like any other UNIX command. It has options, but you never
need to worry about them in most cases. Of course, if you are curious about those options, that leads to our next facet.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Note how much
OptionParser gives us:
- Ability to describe our app, its version, and basic invocation syntax
- Nicely formatted list of options and descriptions
- Ability to accept “negatable” options (we’ll talk about that in a second)
Further, I’ve gone to the trouble to make sure that
--color clearly indicates the acceptable values as well as the default. Finally, I’ve made sure that all options are available in short-form (for easy typing on the command line) and long-form (for clarity when scripting and configuring our app).
Here’s the code that makes this happen (if you aren’t familiar with methadone, the method
on behaves almost exactly like the
on method in
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
description are helpers from methadone (see the intro for more), but note how little code it takes just to make a great and polished UI.
The second part of a helpful app is to include more detailed documentation. For a command-line app, this is expected to be in the form of a man page. If you installed
hl with RubyGems, try this:
You should see a nicely formatted man page (which also happens to be the
README for the github project)! Creating a man page is extremely simple thanks to ronn.
ronn converts Markdown to troff, the format used by the man system. Just add this to your Rakefile:
1 2 3 4 5 6 7 8 9 10
And, your gemspec just needs:
You’ll also need to include the generated file
man/hl.1 in your
files in your gemspec, but if you’re using the gemspec created by Bundler, this happens automatically as long as the file is in source control.
That’s it. Now your app has a great UI and a man page, and all you had to do was drop a few lines of code and write a short Markdown file (which you’d write anyway, since you are making a README, right?).
In addition to being helpful to humans, awesome command-line apps should be helpful to other commands.
Play well with others
An app that “plays well with others” on the command line, basically means that it acts as a filter. Text comes in, gets processed, the processed text goes out. The expectation is that text from any other “well playing” program can be input into our program, and that our program’s output can be piped into another program as input.
Since the purpose of our app is to add ANSI escape codes to the output for assistance with human visual scanning, we can’t claim that our output plays well with others; it’s not designed to. But, we can still play well with the output from other apps.
We saw that
hl was designed to take input from a tool like
hl can also highlight terms from any number of files given to it on the command line. You can do this transparently in Ruby using the awesome ARGF, however Methadone doesn’t support ARGF (a sad fact I learned while writing this app, and something I’ll address in the near future), so here’s how did it (a few comments added to indicate what’s going on):
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Again, ARGF handles this transparently, but the point is, we want the standard input and a provided list of files to be treated the same by our program, and this is how I did it.
Since our app is similar in concept to grep, I thought it would be nice if users familiar with grep could be instantly familiar
Delight Casual Users
This is a “level up” from “being easy to use”. The idea behind the term “delight” is to provide a level of polish and attention to detail that your users will appreciate if they’re observant, but hopefully not even notice, because your app “just works”.
grep, is used for filtering and examining text files, I chose my command-line options to match
grep’s where i could. Initially, I had the short-form of
-i. When I later added the ability to do a case-insensitive match, I realized that
-i is the option to
grep for “case-insensitive”. I quickly changed
--inverse to have
-n as its short-form, and made
--ignore-case the options for case-insensitivity. These are the same values that
grep uses, so a user who might subconciously type
hl -i expecting a case-insensitive match will get it.
Further, I allowed the user to specify the search term either as a command-line argument, or as the argument to
--regexp, which are the option names
grep uses. It’s a basic principle of design that things that are the same should be exactly the same, so I used
grep as my guide when
hl implemented similar features.
Of course, power users love to customize things.
Make Configuration Easy
In the book, I talk about using YAML as a configuration format for an
.rc file. This can be very useful for complex apps, but another technique that’s handy is to allow an environment variable to hold default options.
grep does this via
GREP_OPTS and if you were paying attenion, you noticed this line in
This tells methadone to look at the environment variable
HL_OPTS (as well as the command line) for any options. These options are placed first in
ARGV, essentially like so:
1 2 3
(Note the use of
String to make sure that
nil gets turned into the empty string, saving us an
if statement). Methadone does this before parsing
unshift means that any options the user specifies will come after those in
HL_OPTS and therefore take precendence:
$ export HL_OPTS=--color=cyan $ grep foo some_log.txt | hl --color=magenta
This is the same as
$ grep foo some_log.txt | hl --color-cyan --color=magenta
This is also why I’ve provided the “negatable” forms. Suppose you generally wanted inverse:
$ export HL_OPTS=--inverse
If you wanted to run
hl without inverse, but there was no negatable option, the only way to turn it off would be to unset the environment variable. With the negatable forms, it’s simple:
$ grep foo some_log.txt | hl --no-inverse
Since the user’s command-line options take precedence, things work out, but you can still configure your defaults.
Finally, I’d recommend that you use the long-form options in your configuration. In other words, if you prefer bright and inverted highlights, do this:
$ export HL_OPTS='--inverse --bright'
As opposed to
$ export HL_OPTS=-nb
The second form is more compact, but your configuration is going to be read more than written, and, 6 months from now when you are going through your
.bashrc, you’re going to appreciate seeing things spelled out; you’ll know instantly what the configuration does and don’t have to wonder about what
$ gem install hl $ hl --help
That is all.
1 2 3 4 5 6
It was very easy to do this, although aruba could use a man page for easier reference. I had to jump into its source too many times to get reminded of the syntax of the steps it provides. Aruba also strips out ANSI escape sequences, which made testing
hl a bit tricky. There appears to be an option to prevent this, but I couldn’t get it to work, so I just used Aruba’s internal API:
1 2 3 4 5 6
I still recommend aruba and cucumber, as it forces you to think about how users will use your app first, not how to implement it. In fact, my initial implementation was a big hacky mess of stuff inside the
main block. Once the tests were in place, I refactored it to be a lot cleaner.
Be Easy to Maintain
As I just mentioned, I was able to use my tests to refactor my code. As such, the main block of
hl is pretty simple:
1 2 3 4 5 6 7 8 9 10 11 12 13
This is the sort of logic you want in your
- Handling the keyword-from-argument and keyword-from-command-line-option case
- Simple error checking
- Duping the keyword (since it comes in frozen)
- Calling our
Highlighterclass to do the real work
We defer all non-UI logic to the
Highlighter class. I decided to make each instance of the class able to highlight any files repeatedly based on a configuration, so the constructor takes in the formatting options, and the method
highlight takes the list of filenames and the search term.
The actual highlighting is made possible via lots of list comprehension:
If you aren’t comfortable with this use of chained calls, it can be very powerful. What this does is:
- Map each file to an array of its contents as lines.
[ ['first line of foo\n','second line of foo\n'],['first line of bar\n'],['second line of bar\n']]
- Flatten that array of arrays to just one list of all lines of all files. Our example array becomes:
[ 'first line of foo\n','second line of foo\n','first line of bar\n','second line of bar\n']
- map those lines to the lines with the search term highlighted. Supposing we wanted to highlight the word “line”, our array becomes:
[ 'first \e[33mline\e[0m of foo\n','second \e[33mline\e[0m of foo\n','first \e[33mline\e[0m of bar\n','second \e[33mline\e[0m of bar\n']
- join them all together into one big string
"first \e[33mline\e[0m of foo\nsecond \e[33mline\e[0m of foo\nfirst \e[33mline\e[0m of bar\nsecond \e[33mline\e[0m of bar\n"
Granted, this approach will probably have trouble with extremely large input, but
hl was designed to work with the output of
grep, so hopefully we won’t have too much (I’ve already decided I need it to work with
Breaking the rules
Color and formatting are not typically associated with awesome command-line apps; too much of it makes an app hard to use with other apps. But, the whole purpose of
hl is to colorize output, so for that, I used rainbow, which is a pretty
simple enhancement to
String that allows coloring and formatting. We can see it in action in the
highlight_string method of
1 2 3 4 5 6 7
Each method called on
string is a method provided by Rainbow. These methods return a new string with the appropriate ANSI escape codes added.
Hopefully, you’ve seen that it’s really not that hard to make an awesome command-line app. I was able to write
hl in just a few hours, using TDD and the end result is a highly polished, well-documented, easily installable and maintainable piece of software that will be a part of my command-line arsenal for quite a while. You can do this, too. There’s a lot more detail and in-depth explanations in my book, which you should buy right now :)