In trying to give more control over how Perl programmers test their programs, I've had a couple of people tell me that I can't do that completely since so many people don't use the Test::Builder modules. I don't understand this objection. Why should I worry that someone who chooses to use X doesn't get the features that Y provides?
That being said, I decided to figure out some real numbers. I used CPAN::Mini::Extracted to extract the CPAN libraries and their tests to my hard drive. I then examined the test programs and tried to figure out which test modules programs were using. This is all done heuristically, but from my hand examination of results, it looks pretty close. I accidentally skipped Test.pm my first time through, but here are the top ten testing modules (out of 287 used in almost 60,000 test programs) and the number of times each was used:
Test::More 44461
Test 8937
Test::Exception 1379
Test::Simple 731
Test::Base 316
Test::Builder::Tester 193
Test::NoWarnings 174
Test::Differences 146
Test::MockObject 139
Test::Deep 127
From there, I wanted to figure out the percentage of test programs which used the Test::Builder framework, Test.pm or something else. Again, these are generated heuristicically, but it's interesting:
Uses Count Percent
------------------------------
Test::Builder 47934 81.30%
Non-builder 1107 1.88%
Unknown 642 1.09%
Test.pm 8937 15.16%
It's not perfect, but this tells me that if someone provides new testing functionality and is willing to special case Test.pm, you can capture almost 97% of the testing market on the CPAN.
Have you also tried to look into evaluating the possible relationship between when the module was last released and what it uses? I suspect that more recent distros will tend to use Test::Builder a lot more often.
