Before I start, I just want to say how incredibly thankful I am that I did not take that job at Lehman Brothers. It was soooo tempting.
Test::Aggregate used to not allow 'skip_all' in plans. This was because you would "skip_all" 12,000 tests instead of just the 20 or so in the test program. Very disappointing. It even warned you if it thought you might be using skip_all. I mentioned I might fix it one day, but now I had to fix it (0.34_03 and above), but it turned out to not be easy.
Test::Aggregate uses an Apache::Registry trick to make defer loading some code. We take a small test program like this:
use strict; use warnings; use Test::More 'no_plan'; is 2+2, 4, 'We can add!';
And turn it into something similar to this (trimming some bits):
1 {
2 #################### beginning of aggtests/test.t ####################
3 package aggteststestt;
4
5 sub run_the_tests {
6 local $0 = 'aggtests/test.t';
7
8 # line 1 "aggtests/test.t"
9
10 use strict;
11 use warnings;
12
13 use Test::More 'no_plan';
14
15 is 2 + 2, 4, 'We can add!';
16 }
17 #################### end of aggtests/test.t ####################
18 }
Line 3 guarantees that we're in an encapsulated namespace. Line 6 solves a problem where many tests make assumptions about the contents of $0 and line 8 is a line directive which ensures that errors are reported with the original line number and filename. There's actually a lot more, but this is the core of what we're doing.
The problem was that a "skip_all" in Test::Builder figured out an easy way to halt the tests. It just calls exit and that doesn't work for me. Since these tests are now wrapped in subs, if I wanted to simulate "skip_all" I had to call "return", but as a stand-alone test program, that looks very strange. Nonetheless, I had this at the top of a number of test programs:
if ( $ENV{FAST_TESTS} ) {
if ( $ENV{TEST_AGGREGATE} ) {
Test::Builder->new->skip("Skipping $0 under aggregated fast tests");
return;
}
else {
plan skip_all => 'FAST_TESTS environment variable set';
}
}
else {
plan tests => $num_tests;
}
I couldn't factor that out into a subroutine as that "return" would have to return from an extra stack level. Ugh! (more than once I have failed to refactor code because I need to return or eval code at a different stack level from the current one). However, even using this would throw my "skip_all" warnings in Test::Aggregate.
I was washing dishes on Sunday and thinking "damn it. I wish I had a reliable source filter or Perl 6 grammars. That would make this problem trivial because I ... I ... I..."
Oh. I'm a dumbass.
I'm not using source filters, but I'm already rewriting the damned code! Now the above test script looks a bit like this when rewritten:
1 {
2 #################### beginning of aggtests/test.t ####################
3 package aggteststestt;
4
5 sub run_the_tests {
6 local $0 = 'aggtests/test.t';
7
8 AGGTESTBLOCK: {
9 if ( my $reason
10 = $Test::Aggregate::Builder::SKIP_REASON_FOR{aggteststestt} )
11 {
12 Test::Builder->new->skip($reason);
13 last AGGTESTBLOCK;
14 }
15 # line 1 "aggtests/test.t"
16
17 use strict;
18 use warnings;
19
20 use Test::More 'no_plan';
21
22 is 2 + 2, 4, 'We can add!';
23 }
24 }
25 #################### end of aggtests/test.t ####################
26 }
Ugly, but you should never see this code anyway. Problem solved. (I wanted to put the "last AGGTESTBLOCK" in the Test::Builder::plan subroutine, but because it's called at compile-time, the AGGTESTBLOCK hasn't finished parsing, so &plan cannot "last" out of a block it can't see).
And that ugly if/else "FAST_TESTS" block above? Now it looks like this:
# If it's not in a BEGIN block, it will run too late for
# the &run_the_tests call to see it.
BEGIN {
plan $ENV{FAST_TESTS}
? (skip_all => 'FAST_TESTS environment variable set')
: (tests => $num_tests);
}
All things considered, I think this has been a good day's work.