Here's my Build.PL script from Data::Record:
#!/usr/bin/perl -w
use strict;
use Module::Build;
my $builder = Module::Build->new(
module_name => 'Data::Record',
license => 'perl',
dist_author => 'Curtis "Ovid" Poe ',
dist_version_from => 'lib/Data/Record.pm',
requires => {
'Sub::Uplevel' => 0.09,
},
build_requires => {
'Test::Exception' => 0.21,
'Test::More' => 0.6,
},
add_to_cleanup => ['Data-Record-*'],
create_makefile_pl => 'traditional',
);
$builder->create_build_script();
See anything strange about it? Probably not. You probably aren't familiar with this module or its internals so you wouldn't know that it does not use Sub::Uplevel. Why do I include that? Because I am sick and tired of my modules failing tests because it's not there. You see, earlier versions of Test::Exception used to list Sub::Uplevel in the "build_requires" section of Build.PL. A bug in either Module::Build or CPANPLUS caused this dependency to not be picked up and tests would fail.
The latest versions of Test::Exception's Build.PL script have everything in the "requires" section to get around this. Still, despite this change having been made, my requiring this version of Test::Exception and me explicitly listing Sub::Uplevel as a requirement, I still have a reported failure for Data::Record. It appears to be related to Sub::Uplevel not being found but I can't tell and I've no idea what, if anything, to fix. I suspect this is not my fault.
Wait, it gets worse! The latest problem is five reported failures in my AI::NeuralNet::Simple module. It's a simple module, but it's my first module with C code and as such, it's a point of pride. The previous version has a failure reported because ... anybody, anybody? Bueller? Yeah, because of Sub::Uplevel. So I uploaded a new version with this module listed as a prerequisite and I included POD and POD coverage tests as a bonus. Now I get five failures because Parse::RecDescent is not installed. What? I don't use that module, either. As it turns out, I use Inline::C which does use Parse::RecDescent and correctly lists that module as a dependency. Unfortunately, the lastest CPANPLUS bug shows that CPANPLUS isn't tracking dependencies correctly. It's kind of looking like the bug was in CPANPLUS all the time.
I used to be proud of the fact that none of my modules failed their tests. Today, I get plenty of failures. Maybe they're from Module::Build, maybe they're from CPANPLUS, sometimes they're from obscure bugs with Scalar::Util on certain versions of Perl. Rarely do I get failures that I caused. This is frustrating to put so much work into open-source only to see this.
-sam
Re:Just forget about the test reports
Ovid on 2005-09-28T17:38:01
That sounds nice, but false negatives are as bad as false positives. People either have to dig in and find out if it's really a failure or they learn to ignore failures, force the install and then complain when the module doesn't work. It's very important that this issue get resolved.
Of course, there's yet another class of folks: those who won't install anything with test failures reported. If I have a choice amongst several modules, I don't want to dig through all of the test reports. As a result, I've probably installed modules that aren't the best choice for the job but I only have so much time in the day.
Re:Just forget about the test reports
perrin on 2005-09-29T17:37:48
Do people really pay attention to those test reports? I seem to remember them always being pretty bogus, with lots of failure reports for popular modules that obviously work. In my opinion, there isn't enough agreement about the purpose and extent of unit tests on CPAN modules for the test reports to be meaningful when choosing a module.Re:Just forget about the test reports
Ovid on 2005-09-29T17:51:34
So are you suggesting that having useful test reports is not a worthwhile goal? I don't think you are.
Re:Just forget about the test reports
perrin on 2005-09-29T17:59:45
I'm suggesting that having useful test reports is not an achievable goal, given the current lack of rules for CPAN tests. If you ask ten different authors what the tests on CPAN modules are for, you'll get ten different answers. Many don't intend for end users to run them at all. And there isn't any accepted way to say "this forking module should not be auto-tested on Windows, since it isn't intended to work there." I think the automated testing of all CPAN modules was a neat idea, but ultimately yields poor results, and it has been that way since long before Module::Build and CPANPLUS existed.