In trying to give more control over how Perl programmers test their programs, I've had a couple of people tell me that I can't do that completely since so many people don't use the Test::Builder modules. I don't understand this objection. Why should I worry that someone who chooses to use X doesn't get the features that Y provides?
That being said, I decided to figure out some real numbers. I used CPAN::Mini::Extracted to extract the CPAN libraries and their tests to my hard drive. I then examined the test programs and tried to figure out which test modules programs were using. This is all done heuristically, but from my hand examination of results, it looks pretty close. I accidentally skipped Test.pm my first time through, but here are the top ten testing modules (out of 287 used in almost 60,000 test programs) and the number of times each was used:
Test::More 44461 Test 8937 Test::Exception 1379 Test::Simple 731 Test::Base 316 Test::Builder::Tester 193 Test::NoWarnings 174 Test::Differences 146 Test::MockObject 139 Test::Deep 127
From there, I wanted to figure out the percentage of test programs which used the Test::Builder framework, Test.pm or something else. Again, these are generated heuristicically, but it's interesting:
Uses Count Percent ------------------------------ Test::Builder 47934 81.30% Non-builder 1107 1.88% Unknown 642 1.09% Test.pm 8937 15.16%
It's not perfect, but this tells me that if someone provides new testing functionality and is willing to special case Test.pm, you can capture almost 97% of the testing market on the CPAN.
