Hacking mock: Mock.assert_api(...)

Mock is a great module to use in testing, I use it pretty much all the time. But one thing I have nerver felt great about is the syntax of it's call_args (and call_args_list): it is a 2-tuple of the positional arguments and the keyword arguments, e.g. (('arg1', 'arg2'), {'kw1': None, 'kw2': None}). This does show you exactly how the mock object was called, but the problem I have is that it's more restrictive then the signature in python:

def func(foo, bar, baz=None):
    pass

func(0, 1, 2)
func(0, 1, baz=2)
func(0, bar=1, baz=2)
func(foo=0, bar=1, baz=2)

In this example all the calls to func() are exactly the same from python's point of view. But they will all be different in mock's call_args attribute. To me this means my test will be too tightly coupled to the exact implementation of the code under test. This made me wonder how it could be better.

Firstly I started out writing a function which would take the call_args tuples and know the function signature. This obviously isn't very nice as you need a new function for each signature. But this lead me on to adding the .assert_api() method to the mock object itself. I've been using this method for a while and am still not disappointed in it, so tought I should write about it. Here's how to use it:

mobj = mock.Mock(api='foo,bar,baz')
do_stuff()
mobj.assert_api(foo=0, bar=1, baz=2)

It seems to me this is a fairly good compromise to an extra method on the mock object (and attribute, not shown) and nice concise way of asserting if a function was called correctly.

There are a number of side effects, mainly due to my implementation. The major one is that it consumes .call_args_list! This means each time you call .assert_api() the first item disappears from .call_args_list. Again, I like this as it allows me easily to check multiple calls by just doing multiple .assert_api() calls.

Another side effect, of less importance, is a new attribute api on
the mock object, I can imagine people disliking that one. But it's never been in my way and means you can just assign to it instead of using the keyword argument when creating the mock. I find this handy in combination with patch decorators where the syntax to fine-tune the mock is rather heavy to my liking.

The last strange thing is .assert_api_lazy(), which is just a horribly bad name. It ignores any arguments which are present in the call on the mock object, but where not passed in as part of the api='...' parameter. It's effectively saying you only want to check a few of the arguments and don't care about the others.

Finally here is the code, for simplicity here implemented (and unchecked) as a subclass of Mock:

class MyMock(Mock):
    def __init__(api=None, **kwargs):
        Mock.__init__(self, **kwargs)
        self.api = api

    def _assert_api_base(self, **kwargs):
        """Return call_kwargs dict for assert_api and assert_api_lazy

        WARNING, this consumes self.call-args_list
        """
        if self.call_args_list:
            call_args, call_kwargs = self.call_args_list.pop(0)
        else:
            raise AssertionError('No call_args left over')
        call_args = list(call_args)
        if self.api is None:
            raise AssertionError('self.api is not initialised')
        for p in self.api.split(','):
            if call_args:
                call_kwargs[p] = call_args.pop(0)
        return call_kwargs

    def assert_api(self, **kwargs):
        """WARNING, this consumes self.call_args_list"""
        call_kwargs = self._assert_api_base(**kwargs)
        assert kwargs == call_kwargs, \
            'Expected: %s\nCalled with: %s' % (kwargs, call_kwargs)

    def assert_api_lazy(self, **kwargs):
        """WARNING, this consumes self.call_args_list"""
        call_kwargs = self._assert_api_base(**kwargs)
        for k in call_kwargs.copy().iterkeys():
            if k not in kwargs:
                del call_kwargs[k]
        assert kwargs == call_kwargs, \
            'Expected: %s\nCalled with (truncated): %s' % (kwargs, call_kwargs)

I'd be interested in feedback! It's definitely not perfect, but I do like the syntax of m=Mock(api='...'); ...; m.assert_api(...). One problem I can think of is that it won't deal gracefully with default arguments on the api yet, e..g.:

def func(foo, bar=42):
    pass

func(foo, 42)
func(foo)

These two calls are identical, but .assert_api() won't see them as the same. This hasn't bothered me yet, which is why I haven't looked into it. But I guess it should be considered for a general-purpose implementation of this idea.