All the Perl that's Practical to Extract and Report

Circular references and memory leaks are dead!

pierre on 2003-11-08T16:56:04

At work, we have a big application on production under mod_perl and Pipeline.pm which is leaking. And when I mean leaking, I mean it is a massive flood.
Something had to be done about it.

I tried several modules available. Devel::Leak and Devel::LeakTrace. They sound interesting, but I already knew there was a memory leak. And Devel::LeakTrace depends on glib and doesn't install on my box.

Any way, I figured out what I really needed. I want to scan variables for circular references and have a report of where they are. If I can obtain a reference to a circular refence somewhere in perl, then I can dump it and I can figure out what it is. So I released to CPAN Data::Structure::Util which does just that. You give it a data structure and here you go, it tells if there is a circular reference and where it is. I hacked a script to scan through all the global variables and I could find in 5mn exactly where was our memory leak! We fixed it, then I redid the same process and sure enough we found another circular reference generating a memory leak. Then the 3rd time, I found a circular reference in Template Toolkit. But this one is harmless, since TT does break the circular reference when the object is not needed. So it doesn't generate memory leak.
We have no more memory leak in our code (or at least nothing significant)!

I have the feeling we are about to fix many memory leak in CPAN modules in the near future :-)

Well, mostly dead at least ;)

autarch on 2003-11-08T18:43:02

Please see the bug report I filed on rt.cpan.org for this functionality. The circular ref detection can be fooled by a combination of strong and weak references to the same thing. If it sees the weak reference first, it will not detect any other strong references.

If you have some code with a lot circular refs and are trying to get rid of them all through the use of weak refs, this can be a problem since you might miss some strong refs but still think you're done.

Re:Well, mostly dead at least ;)

pierre on 2003-11-08T19:01:32

Hi, I've replied on CPAN.

Thanks