All the Perl that's Practical to Extract and Report

Devel::Recreate

Ovid on 2007-08-12T14:17:33

There are plenty of ways in Perl for the source code of a package to not match the "in memory" version of that code. Usually this is not a problem, but when it causes bugs, it can be very frustrating to debug. Here's one time where it caused me to waste over an hour of debugging:

use Test::More 'no_plan';

BEGIN {
    use_ok 'Some::Package';
}
# many tests
ok foo($x), '... and foo($x) should return true';

What happened was that the use_ok failed, but since I forgot to put an or die modifier after it, I didn't notice that the use statement failed. Many tests scrolled past and I couldn't figure out why my final test was failing.

This can fail when you try to do something like this:

eval "use Some::Package";
my $object = Some::Package->new;

If we forget to check whether or not the eval succeeded, Perl attempts to compile the code and gives up when it can't. This can leave you with what is effectively a partially compiled namespace and strange bugs abound (I was getting a segfault).

To find out if what's in memory matches what you think it should be, I'm writing Devel::Recreate:

use Devel::Recreate 'recreate';
print recreate('Some::Package');

Due to the dynamic nature of Perl, this is very tough to do. Also, strange bugs have been cropping up (some of which I know are fixed in 5.9). Currently, I have it listing the variables defined and the subroutines defined. Now I'm working on the use and require statements. It should probably always be considered experimental as there are too many corner cases I don't know how to work out. I haven't figured out how to capture code outside of subroutines. BEGIN, END, INIT and CHECK blocks don't seem to be coverable (they're not really subroutines), and I can't distinguish between fully qualified package variables and variables declared with our.

All things considered, I'm not sure if this code will be useful or not, but it's an interesting tour of strange bits of Perl.

Perl_(begin|check|init|end)av[]

jjore on 2007-08-12T15:55:29

Your BEGIN, CHECK, INIT, and END blocks are present in the arrays Perl_(begin|check|init|end)av[]. You will need to set some variable like Perl_savebegin or somesuch to have anything to look at in Perl_beginav[].

There might not be an array for UNITCHECK.

Manip::END lets you muck with END blocks from perl-space. There's another one which does other blocks but I forget its name.

Re:Perl_(begin|check|init|end)av[]

Ovid on 2007-08-13T07:45:01

Aw, crap. My XS skills are week. However, the code itself doesn't look too difficult. I'll play with it. Right now it appears to work, but the code itself isn't too useful because of the 'naked' code reference:

#!/usr/bin/perl -l

use strict;
use warnings;

use Data::Dumper;
use Manip::END;
use Sub::Information;

{
    package Foo;

    END { print 'end!' }
}

my $ends = Manip::END->new;
print Dumper($ends);
print $ends->[0]->();
print inspect( $ends->[0] )->code;

That produces:

$VAR1 = bless( [
                 undef
               ], 'Manip::END' );

end!
1
Use of uninitialized value in string eq at /usr/local/share/perl/5.8.8/Sub/Information.pm line 246.
Bizarre copy of CODE in sassign at /usr/share/perl/5.8/Carp/Heavy.pm line 39.
end!

(Where does that '1' come from?)

You can't assign the code ref to anything and it's virtually useless for my needs. If I were to somehow store this in a scalar and return an array of scalars, it sounds like it would be more useful. I think what I really need is something like this pseudo-code:

AV *get_end_array() {
    int i;
    AV *end_array;
    SV **elem;
    SV *scalar_with_code_ref;

    if (!PL_endav) {
        PL_endav = newAV();
    }

    av_extend( end_array, av_len(PL_endav) );
    for ( i = 0; i < av_len(PL_endav); i++ ) {
        elem = av_fetch(PL_endav, i, 0);

        // store elem in scalar_with_code_ref
        av_store( end_array, i, scalar_with_code_ref );
    }

    return end_array;
}

Unfortunately, my knowledge of perlguts is even worse than my knowledge of C. I'm sure the above is broken (well, I know it is because of the comment) :)

Re:Perl_(begin|check|init|end)av[]

chromatic on 2007-08-13T17:51:05

(Where does that '1' come from?)

It's the return value of the print in the END block, isn't it?

Re:Perl_(begin|check|init|end)av[]

Ovid on 2007-08-13T18:06:23

D'oh!

Naming

Aristotle on 2007-08-12T17:11:38

What’s next? Devel::KickBack? Devel::R’and’R?

:-)

Re:Naming

Ovid on 2007-08-13T07:03:29

Next is Devel::Procreate :)

For the record, I hate the Devel::Recreate namespace. Suggestions welcome.

Hmmm...

btilly on 2007-08-13T06:00:31

Sounds like the wrong solution. There are lots of valid reasons that the module in memory might not match the one on disk. For instance someone could have created new functions by assigning to a typeglob or calling eval.

For most cases, a module similar to Apache::Reload is right. Put in a check for which modules have changed on disk since you loaded them, then reload them.

That doesn't help you in the case you have. But it is easy to create a module that redefines use and require. If you try to use or require another module after that, and the use or require exits with an exception, then before rethrowing the exception you replace all functions in that package with functions that will throw an exception explaining that use failed. If you want to be clever, you can also look through the exports (assuming use of Exporter) and replace exported functions and variables with similarly unusable stubs. In the situation that you describe, the failed use will then be impossible to ignore.

Or you could just take that use out of its eval. What is the eval buying you? Done right, a clearer error message upon failure. But a fatal error there is still pretty easy to debug. So just let the natural exception speak for itself!

Re:Hmmm...

Ovid on 2007-08-13T07:02:32

The above example is only one of many. If a module is not behaving the way I expect it to, particularly if "deep magic" is involved, than it's quite reasonable for me to want to see what's actually loaded in memory instead of what I think was loaded. Just a few areas where this would be helpful:

  • Auto-generated classes (such as the many auto-generated classes that ORM folks are fond of)
  • Method generators (Class::Delegator, Class::BuildMethods)
  • Moose!
  • Source filters (right now, debugging them is a pain in the ass)

Basically, any area where code is dynamically generated means that what's loaded in memory has a good chance of not matching the developer's expectations. When this happens and you're debugging, you want to see what that code looks like.

Re:Hmmm...

btilly on 2007-08-13T08:09:58

I misunderstood. I thought you were proposing this module as a solution to help make sure that what was loaded is the same as the module on disk. You're not. What you're actually proposing doesn't have a simple solution.

Carry on. :-)

Re:Hmmm...

Ovid on 2007-08-13T08:20:19

Yes, I'm convinced that my module could only be used as a rough debugging guide. There are too many limitations that I cannot figure out how to get around, so it should probably only be a tool that developers can use if they read the docs carefully and really understand what its limitations are.

Re:Hmmm...

Aristotle on 2007-08-13T08:34:32

Actually I don’t think it is a bad idea at all. You are right that this is not applicable in all cases in Perl because Perl was not designed to facilitate such usage. However, think Smalltalk, where a lot of “magic” is no problem because what you are browsing is the live in-memory representation of the program and so you see what it looks like after the meta-program parts have taken place.

This is just the same thing for Perl. Of course it’s not going to work nearly as well in Perl… but it’s worth a try anyway.