The Problem

I recently had to write a few command line applications of the form "command [options] args" that did some stuff, maybe printed a few things on screen and exited with a certain exit code. Nothing weird here.

These apps where part of a larger server system however and needed to use some of the modules from these servers for some of their work (in the name of code reuse obviously). A little later these apps would look nicer when they are separated out into their own modules as well (all hail code reuse again the apps can share code) and now it is really a short step to wanting to use some of the more general modules of the apps in the server.

I'm not sure that last step was very important, I think it all started when the app was split up in modules. But the last one made it very obvious: you can't just print random stuff to the user and decide to sys.exit() the thing anywhere you want. You want the code to behave like real modules: throw exceptions and not print anything on the terminal. That's not all, you also want to write unit tests for every bit of code too. Ultimately you need one main routine and you want to test that too, so even that can't exit the program.

The Solution

Executable Wrapper

The untestable code needs to remain to an absolute minimum. Code is untestable (ok, there are work arounds) when it sys.exit()s so I raise exceptions instead. I defined exceptions as such:

class Exit(Exception):
   def __init__(self, status):
       self.status = status

   def __str__(self):
       return 'Exit with status: %d' % self.status

class ExitSucess(Exit):
   def __init__(self):
       Exit.__init__(self, 0)

class ExitFailure(Exit):
   def __init__(self):
       Exit.__init__(self, 1)

This allows for a very small executable wrapper:

#!/usr/bin/env python

import sys
from mypackage.apps import myapp

try:
        myapp.main()
except myapp.Exit, e:
        sys.exit(e.status)
except Exception, e:
        sys.stderr.write('INTERNAL ERROR: ' + str(e) + '\n')
        sys.exit(1)

The last detail is having main() defined as def mypackage.myapp.main(args=sys.argv) for testability, but that's really natural.

Messages for the user

These fall broadly in two categories: (1) short warning messages and (2) printing output. The second type is easily limited to a few very simple functions that do little more then just a few print statements, help() is an obvious example. For the first there is the logging module. In our case the logging module is used almost everywhere in the server code anyway, but even if it isn't it is a convenient way to be able to silence the logging. It's default behaviour is actually rather useful for an application, all that's needed is something like:

import logging

logging.basicConfig(format='%(levelname)s: %(message)s')

The lovely thing about this that you get --verbose or --quiet almost for free.

Mixing it together

This one handles fatal problems the program detects. You could just do a logging.error(msg) followed by a raise ExitFailure. But this just doesn't look very nice, certainly not outside the main app module (mypackages.apps.myapp in this case). But a second option is to do something like

 raise MyFatalError, 'message to user'

And have inside the main() another big try...except block:

try:
        workhorse(args)
except FatalError, e:
        sys.stderr.write('ERROR: ' + str(e) + '\n')
        raise ExitFailure

Just make sure FatalError is the superclass of all your fatal exceptions and that they all have a decent __str__() method. The reason I like this is that it helps keeping fatal error messages consistent wherever you use them in the app, as all the work is done inside the __str__() methods.

One final note; when using the optparse module you can take two stances: (1) "optparse does the right thing and I don't need to debug it or write tests for it" or (2) "I'm a control freak". In the second case you can subclass the OptionParser and override it's error() and exit() methods to conform to your conventions.