devork

E pur si muove

Finding memory leaks in Python extension modules

Saturday, December 13, 2008

Valgrind is amazing, even if you've never used it before it's basic usage is really simple:

$ valgrind --tool=memcheck --leak-check=full /usr/bin/python ./test.py

This is enough to point you at the places in your extension module where you allocated stuff that didn't get freed later. This is a massive timesaver with looking over the entire source file again to find out where you made your mistakes.

I must admit that the extension module in question uses malloc(3) and free(3) directly instead of allocating on the Python heap using PyMem_Malloc() and PyMem_Free(), so I don't know if that would make it harder to find the leaks. I can imagine that in that case the "blocks definitely lost" list might point to somewhere in Python's source files instead of your own source files, but I don't know.

Saturday, December 13, 2008 | Labels: |

2 comments:

Anonymous said...

Note that this only finds leaks of your objects. If you leak Python objects (eg ints, string etc) then those won't be found.

I compile a debug Python with all Python's memory tricks turned off. Suppressions with a normal Python won't help that much. See the comments at the top of this file:

http://code.google.com/p/apsw/source/browse/apsw/trunk/tools/valgrind.sh

Another thing I do is run the tests multiple times (call unittest.main() in a loop). Then compare the leak check output with 5 vs 10 iterations. If the numbers aren't the same then you can find where the leaks are with a diff.

New comments are not allowed.

Subscribe to: Post Comments (Atom)