devork

E pur si muove

Decorators specific to a class

Wednesday, December 30, 2009

My gut tells me it's horribly wrong but I am failing to formulate a decent argument, let me show an example what I mean (somewhat contrived):

def deco(f):
    def wrapper(self, *args, **kw):
        with self.lock:
            f(self, *args, **kw)

class Foo:
    def __init__(self):
        self.lock = threading.Lock()

    @deco
    def method(self):
        pass

Here the decorator knows something about the arguments of the function it will end up calling, the first argument is "self" and it is an object with a "lock" attribute which is a context manager. Somehow I feel like that's more knowledge about the wrapped object then a decorator should have. It just is an indirection of logic my bain doesn't cope with.

There are obviously places where I could construct something like this. But I do never naturally think of doing it that way, I always end up with some other way which I find more elegant and I think the resulting logic is easier to follow. It's just that whenever I encounter code like this my brain starts hurting and I'm not sure I have a decent argument to tell people writing code like this off (you can hardly regard "it's a level of logic indirection that makes my brain hurt" as an argument).

Calling COM methods in C

Saturday, December 26, 2009

So Windows has this strange thing called COM. It allows you to share objects between unrelated processes and languages, a bit like CORBA in that sense, only very different. Anyway, if you'd like to get information out of this other Windows thing called WMI (which provides a lot of information you can't get hold of using native APIs, at least not if you don't digg into unpublished internals) then you've got to use COM.

But Microsoft seems to have decided some long time ago that C++ is such an amazing language (because if it's got ++ in the name it must be better then say just C, you know) that it solves all of the world's problems. So obviously that is the language you favour when designing APIs, meaning you end up with very convoluted APIs when using good old plain C. And obviously there is no need to document the way you do things in C because no one would use it. Anyway, my conclusion is that COM is crazy and I can only assume that .NET must somehow make this a bit easier (just like using WMI from Python is easy with Mark Hammond's pythoncom and Tim Golden's wmi module) which is probably the reason that C# is popular among developers who choose Windows as their platform.

Back to the point however: how do you call a method on a COM object using C? Microsoft will always show instance->Method() in their examples, but in C it's not that much different:

instance->lpVtbl->Method(instance)

There is also supposed to be a way to use some macro's using the class name of the object joined up with the method name. For this you need to define the COBJMACROS before your #includes and then you can call:

ClassName_Method(instance)

Only I can't get that way to work. No clue why.

For completeness here is the full example of the WMI local computer example in the MSDN library, ported to C. For extra fun the error handling is done by setting Python exceptions, but that shouldn't confuse things. It also prints the result string in both unicode and ascii, from the unicode string you could have easily made a PyUnicodeObject. There's probably many ugly things in there as I'm really not a win32 developer. It's also example code and not what I'd take into production (e.g. I think the looping over the results is broken but I'm sticking close the the MSDN example).

static int
example(void)
{
    HRESULT hr;
    IWbemLocator *pLoc;
    IWbemServices *pSvc;
    IEnumWbemClassObject *pEnumerator;
    BSTR bstr, bstr2;

    hr = CoInitializeEx(0, COINIT_APARTMENTTHREADED);
    if (hr == RPC_E_CHANGED_MODE)
        hr = CoInitializeEx(0, COINIT_MULTITHREADED);
    if (hr != S_OK && hr != S_FALSE) {
        PyErr_Format(PyExc_WindowsError,
                     "Failed to initialise COM (HRESULT=0x%x)", hr);
        return -1;
    }
    hr = CoInitializeSecurity(NULL, -1, NULL, NULL,
                              RPC_C_AUTHN_LEVEL_DEFAULT,
                              RPC_C_IMP_LEVEL_IMPERSONATE,
                              NULL, EOAC_NONE, NULL);
    if (FAILED(hr)) {
        PyErr_Format(PyExc_WindowsError,
                     "Failed to initialise COM security (HRESULT=0x%x)", hr);
        CoUninitialize();
        return -1;
    }
    hr = CoCreateInstance(&CLSID_WbemLocator, 0, CLSCTX_INPROC_SERVER,
                          &IID_IWbemLocator, (LPVOID *) &pLoc);
 
    if (FAILED(hr)) {
        PyErr_Format(PyExc_WindowsError,
                     "Failed to get IWBemLocator (HRESULT=0x%x)", hr);
        CoUninitialize();
        return -1;
    }
    bstr = SysAllocString(L"ROOT\\CIMV2");
    if (bstr == NULL) {
        PyErr_SetString(PyExc_WindowsError, "BSTR allocation failed");
        pLoc->lpVtbl->Release(pLoc);
        CoUninitialize();
    }
    hr = pLoc->lpVtbl->ConnectServer(pLoc, bstr, NULL, NULL,
                                     NULL, 0, NULL, NULL, &pSvc);
    SysFreeString(bstr);
    if (FAILED(hr)) {
        PyErr_Format(PyExc_WindowsError,
                     "Localhost connection for WMI failed (HRESULT=0x%x)", hr);
        pLoc->lpVtbl->Release(pLoc);
        CoUninitialize();
        return -1;
    }
    hr = CoSetProxyBlanket((IUnknown*)pSvc, RPC_C_AUTHN_WINNT, RPC_C_AUTHZ_NONE,
                           NULL, RPC_C_AUTHN_LEVEL_CALL,
                           RPC_C_IMP_LEVEL_IMPERSONATE, NULL, EOAC_NONE);
    if (FAILED(hr)) {
        PyErr_Format(PyExc_WindowsError,
                     "Failed to set ProxyBlanket (HRESULT=0x%x)", hr);
        pSvc->lpVtbl->Release(pSvc);
        pLoc->lpVtbl->Release(pLoc);
        CoUninitialize();
        return -1;
    }
    bstr = SysAllocString(L"WQL");
    if (bstr == NULL) {
        PyErr_SetString(PyExc_WindowsError, "BSTR allocation failed");
        pSvc->lpVtbl->Release(pSvc);
        pLoc->lpVtbl->Release(pLoc);
        CoUninitialize();
        return -1;
    }
    bstr2 = SysAllocString(L"SELECT * FROM Win32_OperatingSystem");
    if (bstr2 == NULL) {
        PyErr_SetString(PyExc_WindowsError, "BSTR allocation failed");
        SysFreeString(bstr);
        pSvc->lpVtbl->Release(pSvc);
        pLoc->lpVtbl->Release(pLoc);
        CoUninitialize();
        return -1;
    }
    hr = pSvc->lpVtbl->ExecQuery(pSvc, bstr, bstr2,
                                 WBEM_FLAG_FORWARD_ONLY |
                                 WBEM_FLAG_RETURN_IMMEDIATELY, 
                                 NULL, &pEnumerator);
    SysFreeString(bstr);
    SysFreeString(bstr2);
    if (FAILED(hr)) {
        PyErr_Format(PyExc_WindowsError, "WMI query failed (HRESULT=0x%x)", hr);
        pSvc->lpVtbl->Release(pSvc);
        pLoc->lpVtbl->Release(pLoc);
        CoUninitialize();
        return -1;
    }
    {
        IWbemClassObject *pclsObj;
        ULONG uReturn;
        VARIANT vtProp;       
            
        while (pEnumerator) {
            hr = pEnumerator->lpVtbl->Next(pEnumerator, WBEM_INFINITE,
                                           1, &pclsObj, &uReturn);
            if(uReturn == 0)
                break;
            hr = pclsObj->lpVtbl->Get(pclsObj, L"Name", 0, &vtProp, 0, 0);
            wprintf(L"XXX OS Name: %s\n", vtProp.bstrVal);
            {
                /* XXX Need error checking in here */
                /* Allocating the UTF-16 string size since that will be at
                 * least double the ASCII size, which is fine. */
                char *prop;
                int r;

                prop = psi_malloc(SysStringByteLen(vtProp.bstrVal));
                /* if (prop == NULL) */
                r = WideCharToMultiByte(20127, 0, vtProp.bstrVal, -1, prop,
                                        SysStringByteLen(vtProp.bstrVal),
                                        NULL, NULL);
                /* if (!r) */
                printf("XXX OS Name: %s\n", prop);
            }
            VariantClear(&vtProp);
            pclsObj->lpVtbl->Release(pclsObj);
        }
    }
    pEnumerator->lpVtbl->Release(pEnumerator);
    pSvc->lpVtbl->Release(pSvc);
    pLoc->lpVtbl->Release(pLoc);
    CoUninitialize();
    return 0;
}

Setting descriptors on modules

Wednesday, December 23, 2009

This counts for one of the more crazy things I'd like to do with Python: insert an instance of a descriptor object into the class dict of a module.

While you can get hold of the module type class by using the __class__ attribute of any module or use types.ModuleType you obviously can't do this since the __dict__ of the module class is actually a DictProxy and hence immutable. Which I think is rather sad this time round.

My use case is to be able to set an attribute on a module that would lazily evaluate a sort of semi-singleton. Suppose you have the instance of your application and to make it available to other modules you place the instance in a module attribute. You want it to be available since it has references to useful global things like the configuration instance in use etc (and you hate having singletons for all these useful things). To now get this application instance from another module, without getting it passed in with some sort of dependency-injection (which can result in intangible spaghetti all too easily), you can now get hold of it like this:

import app_package.app_module
app_package.app_module.app_instance # the instance

But what you very likely can't do is this:

from app_package.app_module import app_instance

The reason is that import statements are usually at the top of modules and thus you will very likely execute this import statement before the app_instance existed and you will get None (I will anyway, since I initialised that attribute with None before the app gets instanced). And obviously even once the application is instanced and the attribute gets set, I'm still stuck with the reference to None instead of the application instance.

So my reason to want to set a descriptor instance in the module class is so that the descriptor could lazily evaluate the application instance when accessing it. Getting and using the instance would reduce to something like:

from app_package.app_module import app_instance
app_instance # the instance!

Which I think is much cleaner. But for now I'll have to stick with:

from app_package.app_module import get_instance
get_instance() # the instance

Or perhaps Python has another trick up it's sleeve I haven't found yet?

Skipping slow test by default in py.test

Saturday, December 19, 2009

If you got a test suite that runs some very slow tests it might be troublesome to run those all the time. There is of course the "-k" option to py.test which allows you to selectively enable only a few tests. But what I really wanted was a way to have it skip slow tests by default but still allow me to enable slow tests with a command line option.

But this is not impossible, py.test provides a couple of things that make it possible to do this surprisingly easy:

So how do you pull this together? I decided that i want to mark slow tests using the "slow" keyword (@py.test.mark.slow) and skip those tests by default. And the option to enable those tests would be called "--slow". The following conftest.py is staggeringly simple:

import py.test

def pytest_addoption(parser):
    parser.addoption('--slow', action='store_true', default=False,
                      help='Also run slow tests')

def pytest_runtest_setup(item):
    """Skip tests if they are marked as slow and --slow is not given"""
    if getattr(item.obj, 'slow', None) and not item.config.getvalue('slow'):
        py.test.skip('slow tests not requested')

Finding this solution was a little harder however, but essentially it involved nothing more then looking at the pytest_skipping and pytest_mark plugins to figure out what APIs the various objects provide. But it must be said that extending py.test is staggeringly simple, I tend to expect a much higher learning curve when I decide that I want to write a plugin for some tool I use.

setuptools vs distribute

Friday, December 18, 2009

Reinout van Rees:

In case you heard of both setuptools and distribute: distribute fully replaces setuptools. Just use distribute. Setuptools is “maintained” (for various historically dubious values of “maintain”) by one person (whom all should applaud for creating the darn thing in the first place, btw!). Distribute is maintained by a lot of people, so bugs actually get fixed. And “bugs” meaning “it doesn’t break with subversion 1.6 because patches that fix it don’t get applied for half a year”.

Pretty much one of the best descriptions of the differences between setuptools and distribute that I've seen. Not that I use either, but whatever.

update: Obviously this was written many moons ago and a lot has changed in packaging since. And currently (Nov 2013) setuptools is the way to go if you want/need it.

Dictionary entry

Monday, December 07, 2009

innovative company noun 1 a company which can't afford patent lawyers 2 one of the lucky handful companies who are innovative and have money for patent lawyers HELP Not to be confused with most companies lobying in patent litigation.

PS: It's also possible to have companies that can't afford patent lawyers but still don't innovate.

How to drive on a motorway

Sunday, December 06, 2009

If you are driving on a motorway where I am driving too, here are some rules you should follow. I think you should even follow them when I'm not around.

  1. Be sensible, this one overrules all the others
  2. Keep left unless overtaking
  3. KEEP LEFT UNLESS OVERTAKING
  4. KEEP LEFT UNLESS OVERTAKING
  5. Always indicate when changing lanes
  6. Pick a speed and stick to it, avoid speeding up or slowing down whenever possible
  7. Be courteous, even when people make mistakes

It's not that hard, try it and you'll realise traffic would be a lot more fluent if everyone stuck to these rules.

PS: Don't forget to substitute left with right depending on your country.

How to make a crosslinked RS-232 cable

Thursday, December 03, 2009

Because it's easier then buying one

When connecting two computes together (e.g. your laptop to a server or router) to get access to a console you need to use a crosslinked RS-232 cable, usually with two female DE-9 connectors these days. This cable is more commonly known as a null-modem cable for historical reasons.

Of course you can easily buy a cable under the name of "null modem cable", but you're almost guaranteed to get a sloppy made one. In fact I haven't been able to find a decently made one. The problem is that according to the standard the DTR is supposed to be connected to the DSR and CD, but most cables as sold won't connect the carrier detect line. No idea why, perhaps making the bridge is something other connectors don't need and therefore expensive to automate for production?

So to make one yourself you can cut up an existing cable and solder new DE-9 connectors onto it yourself, keeping to the proper schema. It's really not that hard, all you need is a soldering iron and two D-sub 9 pin connectors and corresponding hoods. I'm guessing you could even use cheap CAT-5e networking cable instead of cutting up an expensive pre-made null-modem cable since Sun uses network cables for their serial cables (whith an adaptor).

Now most devices actually seem to ignore the carrier detect line anyway. But in case you are lucky enough to be using IBM's AIX or IVM on one of their System p5 servers you will discover that you are able to log into the service processor with a "normal" (i.e. non-standard) serial cable just fine. You will also be able to perform an installation from that searial cable just fine. But when you then need to access the console of that just installed system you're stuck and just see nothing. Until you use a proper cable that is. (Just for the record, you can most likely telnet or ssh into the system by now, just try it).

And if my love for AIX is not yet clear from the above paragraph let me be a bit more explicit: if you are in a position to choose get something else then AIX. Sun's hardware as well as Solaris are lovely (they ship with a serial cable for instance - as does cisco). Obviously GNU/Linux is a great choice for an OS too. This doesn't mean that I approve of cable vendors not making RS-232 crosslink cables propperly of course.

Subscribe to: Posts (Atom)