E pur si muove

Setting descriptors on modules

Wednesday, December 23, 2009

This counts for one of the more crazy things I'd like to do with Python: insert an instance of a descriptor object into the class dict of a module.

While you can get hold of the module type class by using the __class__ attribute of any module or use types.ModuleType you obviously can't do this since the __dict__ of the module class is actually a DictProxy and hence immutable. Which I think is rather sad this time round.

My use case is to be able to set an attribute on a module that would lazily evaluate a sort of semi-singleton. Suppose you have the instance of your application and to make it available to other modules you place the instance in a module attribute. You want it to be available since it has references to useful global things like the configuration instance in use etc (and you hate having singletons for all these useful things). To now get this application instance from another module, without getting it passed in with some sort of dependency-injection (which can result in intangible spaghetti all too easily), you can now get hold of it like this:

import app_package.app_module
app_package.app_module.app_instance # the instance

But what you very likely can't do is this:

from app_package.app_module import app_instance

The reason is that import statements are usually at the top of modules and thus you will very likely execute this import statement before the app_instance existed and you will get None (I will anyway, since I initialised that attribute with None before the app gets instanced). And obviously even once the application is instanced and the attribute gets set, I'm still stuck with the reference to None instead of the application instance.

So my reason to want to set a descriptor instance in the module class is so that the descriptor could lazily evaluate the application instance when accessing it. Getting and using the instance would reduce to something like:

from app_package.app_module import app_instance
app_instance # the instance!

Which I think is much cleaner. But for now I'll have to stick with:

from app_package.app_module import get_instance
get_instance() # the instance

Or perhaps Python has another trick up it's sleeve I haven't found yet?

Wednesday, December 23, 2009 | Labels: |


holger krekel said...

in app_package/ you could override sys.modules['app_package'] with your own module subclass which can have descriptors. and werkzeug and probably others use this technique, works on many python versions (not sure about IronPython though).

ferringb said...

Few potentials; either

1) use demandload of some variety, and store the singleton in the module namespace per the norm- via delaying the import, you delay the creation. Not great though.

2) shift the singleton onto a class, and use a classproperty to generate the singleton on first access.

Floris Bruynooghe said...

Inserting a module subclass into sys.path seems to work fine. I had experimented with putting a non-module class with a descriptor into sys.modules but did think it was a bit too dirty to risk. But creating a module subclass is an excelent idea. Here the code I tested:

import sys
import types

_app_instance = None

class _MyDesc(object):
    def __get__(self, instance, owner):
        if _app_instance is None:
            raise RuntimeError('app not initialised yet')
            return _app_instance

class _MyModule(types.ModuleType):
    app_instance = _MyDesc()

class TheApp(object):
    def __init__(self):
        global _app_instance
        _app_instance = self

def _replace_module():
    module_name = 'test'
    mod = sys.modules[module_name]
    for name, obj in mod.__dict__.iteritems():
        if name in ('__doc__'):
        setattr(MyModule, name, obj)
    mymod = _MyModule(module_name)
    sys.modules[module_name] = mymod


Floris Bruynooghe said...

Actually, that still doesn't work. When doing "from test import app_instace" the descriptor is invoked instead of just returned as is. So my basic idea of using a descriptor to support lazy "from ... import ..." is broken.

I didn't actually try using apipkg but I think it suffers from the same problem when using the "from ... import ..." form.

ferringb said...

Well... you could always do an object proxy, or make the singleton JIT itself (generally I abuse delayed instantiation object proxies instead of implementing a custom JIT'er each time, but to each their own)...

New comments are not allowed.

Subscribe to: Post Comments (Atom)