devork

E pur si muove

Generating source files in setup.py

Friday, January 09, 2009

I have a Python extension module written in C and one of the weirder things it needs to do is have a number of module constants with their values which are defined (e.g. #define PREFIX_FOO 0x01 etc.) in an external C header file. All the defined names in that header file start with a common prefix so it's easy enough to write a python function that will read the file and spit out the correct C source code that enables me to expose these in my python module. The tricky part however is where to hook this up in the setup.py script.

At first I tried to extend the distutils.command.build_ext.build_ext class to generate this file. That doesn't work however as the distribution is not very happy about having a file listed as required (Extension(sources=['generated_file.c', ...], ...)) which isn't actually there at the time the distutils.dist.Distribution instance is created.

So the two (AFAIK) remaining options are to subclass distutils.dist.Distribution and pass into the setup method: setup(distclass=MyDistribution, ...). Or secondly, don't add the generated source file to the list of required files and created in the extended build_ext command.

For now I've gone for the last option as it seems the more appropriate place to do things (I've overwritten the .initialize_options() method). But I wonder if other more elegant solutions exist?

Friday, January 09, 2009 | Labels: |

1 comments:

Robert Kern said...

numpy.distutils has a mechanism for this (please pardon the formatting):

def gen_source(ext, build_dir):
target = os.path.join(build_dir, 'somefile.c')
# Make parent directories if necessary, check if the file doesn't exist yet, or needs updating, etc.
# Write the file.
open(target, 'w').write(...)

ext = Extension('somefile', sources=[gen_source])

Unfortunately, the support code is fairly large, so I don't think you can just extract it out.

New comments are not allowed.

Subscribe to: Post Comments (Atom)