devork

E pur si muove

Resistance to change

Saturday, January 31, 2009

Why can C developers be happy with using config.h to get constants about where to look for configuration or data files for example, yet Python developers seem to refuse to think of any way they might want to support finding these files in a portable way?

FreeBSD on Virtualbox

Saturday, January 31, 2009

Virtualbox is my desktop virtualisation technology of choice most of the time and I wanted to have a play with FreeBSD. Seems they don't get along tough and you won't get a network connection.

  • Solution 1: Change the network adaptor in Virtualbox to PCnet-PCI II (instead of PCnet-PCI III). I've tried this and it works.
  • Solution 2: I haven't tried this.

While we're on the subject, when running the Ubuntu 8.10 server jeos edition on Virtualbox don't forget to enable PAE in Virtualbox or you get to do a kernel dance using the resque mode of the installer.

datetime.datetime.totimestamp()

Tuesday, January 27, 2009

There exists a datetime.datetime.fromtimestamp() and many a time I've wondered why there is no .totimestamp() equivalent. The answer is that not all datetime objects can be represented by a timestamp. But if you know this restriction and want it anyway, this is the magic:

time.mktime(datetime_object.timetuple())

Would be so cool if the docs actually mentioned this.

PS: Both functions have a utc variant too

Update:

datetime_object.strftime('%s')

This solution does not give you sub-second resolution, but otherwise is rather elegant. Funny thing is that the %s specifier is not documented by the stdlib, but seems to exist on the underlying implementations at least on UNIX.

And for completeness, these issues are being discussed in the bug tracker. See issue2736 and issue1673409.

Resident Set Size (RSS) from /proc/pid/stat

Thursday, January 22, 2009

Most UNIX-like systems have information about processes stored in /proc/pid/ files, so does Linux.

If you would want to get the Resident Set Size (RSS) of a process on Linux you could find this in a number of files:

  • /proc/pid/stat: targetted for scanf(3)
  • /proc/pid/statm: targetted for scanf(3) but just memory information (more detailed)
  • /proc/pid/status: targetted at humans

Oddly enough if you check these three files for the RSS memory you will get different results! It seem both stat and statm have the wrong RSS information. It is a mystery to me why.

Porting a C extension module to py3k

Thursday, January 15, 2009

This was not that hard! First look at what everyone else has to say about this, nice aggregated on the python wiki. It covers a lot.

Now for what I discovered on top of this:

  • When you're defining a new type self->ob_type doesn't exist anymore. This is a problem as you need it in the deallocator for example. The solution is to use Py_TYPE(self). So the deallocator becomes: Py_TYPE(self)->tp_free((PyObject*)self);
  • When you're paranoid and use -Werror -Wredundant-decls you'll notice a duplicate declaration in pyerror.h. Bug filed.
  • The module initialisation is a lot easier then in the official docs. You seem perfectly fine without module state, so all you need to fill out in your struct PyModuleDef is m_base, m_name and m_size. Of course m_doc and m_methods are pretty useful too, but not strictly required. Copy the tutorial here.
    And if you use PyMODINIT_FUNC to declare it all you need to #ifdef is PyInitmymodule(void), PyModule_Create() and the return value.

Generating source files in setup.py

Friday, January 09, 2009

I have a Python extension module written in C and one of the weirder things it needs to do is have a number of module constants with their values which are defined (e.g. #define PREFIX_FOO 0x01 etc.) in an external C header file. All the defined names in that header file start with a common prefix so it's easy enough to write a python function that will read the file and spit out the correct C source code that enables me to expose these in my python module. The tricky part however is where to hook this up in the setup.py script.

At first I tried to extend the distutils.command.build_ext.build_ext class to generate this file. That doesn't work however as the distribution is not very happy about having a file listed as required (Extension(sources=['generated_file.c', ...], ...)) which isn't actually there at the time the distutils.dist.Distribution instance is created.

So the two (AFAIK) remaining options are to subclass distutils.dist.Distribution and pass into the setup method: setup(distclass=MyDistribution, ...). Or secondly, don't add the generated source file to the list of required files and created in the extended build_ext command.

For now I've gone for the last option as it seems the more appropriate place to do things (I've overwritten the .initialize_options() method). But I wonder if other more elegant solutions exist?

Subscribe to: Posts (Atom)