Test::Aggregate

Ovid on 2007-11-26T10:55:16

It's only an experiment and needs lots of work, but Test::Aggregate is now on the CPAN (you'll want the recently uploaded 0.02 version, not 0.01). This module allows you to run and load all of your test programs at once. This means that Perl is only loaded once and your modules are only loaded once. I've borrowed a trick of Apache::Registry to make this work.

Like mod_perl, there are caveats to how you can write your test code (many test programs just work), but you can gain huge improvements in your test suite's run times. Here's a quick test I did at work with just a handful of our 9000+ tests:

$ prove -r t/some/unit_tests/
...
All tests successful.
Files=12, Tests=331,  8 wallclock secs ( 0.08 usr  0.05 sys +  6.05 cusr  0.78 csys =  6.96 CPU)
Result: PASS

$prove agg/aggregate.t
All tests successful.
Files=1, Tests=343,  2 wallclock secs ( 0.06 usr  0.01 sys +  1.65 cusr  0.15 csys =  1.87 CPU)
Result: PASS

In other words, we went from 8 wallclock seconds down to 2 (the differing test counts are due to extra tests being automatically inserted). And these were for some of our fastest tests. I'm sure that our tests which use Catalyst and DBIx::Class will have equally impressive gains, assuming I can get them to run under Test::Aggregate.

Given that our test suite currently takes 22 minutes to run, this could be a huge win.