More than just about any other factor, what I love most about the CPAN is that it solves the cross-platform problem so well.
Rather than try to factor out change by adopting one platform's concept, or going for a lowest-common-denominator approach, it embraces diversity.
Modules like File::HomeDir (which I loved even before I took it over) with it's vastly differing dependencies on different platforms make programming interesting, because they robustly solve the problem of getting the platform out the way.
I can accept that my code might be missing a feature, or have some bug. But I don't want to have to care WHERE I'm writing some bit of code. I just expect my stuff to work everywhere.
And so I tend to spend quite a bit of time keeping my CPAN Testers FAIL counts for modules at zero. And I've done a reasonably good job of it so far, despite the volume of packages I maintain.
But unfortunately, this is starting to become very very hard to achieve.
The problem started when the CPAN Testers modules started to get good enough that people were considering hooking up their bleed tests and developer-release tests and developer versions of Perl to the same reporting mechanisms.
And so now we have this situation where any legitimate production testing results are drowned out in the sea of results from developer releases.
Worse, the TOTAL pass/fail counts are reported inclusive of these developer releases as well, leading the headline search.cpan.org testing results that people rely on when choosing modules to use to be invalid, all because some blead patch a year ago accidentally broke some minor part of Perl, causing half of CPAN to fail to build.
It may be interesting to a small cadre of people SOME of the time, but it's at best not interesting to the public, and at worst highly misleading of the stability of the CPAN.
I say that we've now passed a threshold where this extra testing resolution is causing more harm than good, and I strongly recommend that anyone with a site in the CPAN cloud comprehensively filter out non-production information by default.
Primarily this should mean no more visibility of things like Perl 5.9.1 or 5.11.0 patch 123456 by default, until someone throws some notional "include developer Perls" toggle.
I'm tempted to suggest we also filter out CPAN module developer releases by default. But I have some concerns that if we did this it will discourage awareness and use of developer releases amongst the authors as a whole.
I've also been thinking about this recently, but have yet to think of a suitable solution. While the dev release testing is useful for authors, it isn't for users who don't expect to ever use those releases in a production environment. This applies to past developer releases 5.9.x and 5.7.x, as well as 5.11.x. Incidentally was 5.5.x a developer release?
In the future with the newer system that's in production, it will hopefully be easier to filter these releases out. However, at the moment, the CPAN Testers reports site is static and tells you everything.
I've considered filtering them out for the CPAN Testers Statistics site, or at least providing a side by side comparison in the graphs, but it's the report site that most people look at.
Re:Agreed
Alias on 2008-05-31T07:46:51
Personally, I'm quite happy for the statistics site to keep including developer releases, at least in the headline graphs.
For the version/platform matrix, that's where I'd like to see you remove dev Perls
Since the testers responsible for these dev versions are (a) our #1 tester (by volume) and (b) one of our most respected module authors, perhaps this could be raised on the CPAN Testers Discussion list -- which they both read -- and a proposal for change worked out.
