devork

E pur si muove

Using __getattr__ and property

Friday, June 17, 2011

Today I wasted a lot of time trying to figure out why a class using both a __getattr__ and a property mysteriously failed. The short version is: Make sure you don't raise an AttributeError in the property.fget()

The start point of this horrible voyage was a class which looked roughly like this:

class Foo(object):

    def __getattr__(self, name):
        return self._container.get(name, 'some_default')

    @property
    def foo(self):
        val = self._container.get(foo)
        if test(val):
            return some_helper(val)
        return val

This sees fine enough. Only it turns out that some_helper() raised an AttributeError for some invalid input. Certainly reasonable since it was never meant to deal with incorrect input, that was meant to have been sanitised already (that was a bug in the caller which was actually just an incorrect unittest). The main gotcha was that it seems that python doesn't just check whether a "foo" is present in all the relevant __dict__'s along the mro. Instead it seems to use getattr(inst, "foo") and then delegate to __getattr__() if it gets an AttributeError. Now suddenly finding a bug in some_helper() has turned into a puzzling question as to why __getattr__() was called.

Personally I can't see why it doesn't use the mro to statically look up the required object instead of using the AttributeError-swallowing approach. But maybe there's a good reason.

3 comments:

เสก said...

I have similar problem years ago with IronPython. It seems raising certain Exception inside property getter can cause very ridiculous behavior.
Having the same problem in two different interpreter suggest that this behavior is probably well defined.

deno said...

Hi, so this is because of how property descriptor is implemented. Just how you can return or raise NotImplementedError() in __eq__, __add__ etc., to signal that you don't implement a feature, the exception is being caught by the descriptor. Likewise, AttributeError is property's version of NotImplementedError.

The simplest fix would be to use a try catch, like this:

@property
def foo(self):
try:
some_helper()
except AttributeError as e:
raise RuntimeError from e

You can also put it into your own descriptor, for example:

class AlternativeProperty:

def __init__(self, fget):
self.fget = fget

def __get__(self, obj, objtype=None):
try:
return self.fget(obj)
except AttributeError, e:
raise RuntimeError from e

def __set__(self, obj, value):
raise AttributeError()

then

class Foo:

def __getattr__(self, name):
return "match all getattr"

def _get_foo(self):
raise AttributeError()

foo = FinalProperty(_get_foo)


foo = Foo()

assert foo.abc == "match all from getattr"
assert_raises(RuntimeError, lambda: foo.foo)

Hope this makes it less magical ;)

Shawn Fumo said...

Thanks much for this! I had a proxy class using __getattr__ and added a property and first was afraid that you couldn't do both due to timing of when stuff was called in the process. Turns out I just meant to use self.id instead of self.foo.id (since now I didn't need to reference the wrapped object because of the proxying).

New comments are not allowed.

Subscribe to: Post Comments (Atom)