Finding memory leaks in Python extension modules
Valgrind is amazing, even if you've never used it before it's basic usage is really simple:
$ valgrind --tool=memcheck --leak-check=full /usr/bin/python ./test.py
This is enough to point you at the places in your extension module where you allocated stuff that didn't get freed later. This is a massive timesaver with looking over the entire source file again to find out where you made your mistakes.
I must admit that the extension module in question uses malloc(3) and free(3) directly instead of allocating on the Python heap using PyMem_Malloc() and PyMem_Free(), so I don't know if that would make it harder to find the leaks. I can imagine that in that case the "blocks definitely lost" list might point to somewhere in Python's source files instead of your own source files, but I don't know.
2 comments:
Anonymous said...
Maybe you could use the suppression file provided with python:
http://blogs.gnome.org/jamesh/2008/03/24/python-valgrind/
Anonymous said...
Note that this only finds leaks of your objects. If you leak Python objects (eg ints, string etc) then those won't be found.
I compile a debug Python with all Python's memory tricks turned off. Suppressions with a normal Python won't help that much. See the comments at the top of this file:
http://code.google.com/p/apsw/source/browse/apsw/trunk/tools/valgrind.sh
Another thing I do is run the tests multiple times (call unittest.main() in a loop). Then compare the leak check output with 5 vs 10 iterations. If the numbers aren't the same then you can find where the leaks are with a diff.
New comments are not allowed.