CPAN Testers Stats - March update

barbie on 2007-04-04T13:16:39

For those interested, the latest data for March has been uploaded. Interesting to note that the top 5 report posters have all ramped up their report submissions this month. Chris is steaming ahead, with nearly twice as many submissions as his nearest rival. Recent recruit to CPAN Testing, Dave Cantrell, has made a dramatic climb up the leader board, and is sure to feature in the top 10 next month.

Looking at the new table Failure Counts, I've been alarmed at how many distributions have recently been uploaded and failed on all platforms. Some are minor bugs, which should have been spotted by the author before uploading, but some look to be a bit more problematic. However, 43 out of 100 in the above table, were from uploads made in 2007 and all produced FAIL or UNKNOWN test reports, with no PASSes or NAs at all. I guess this was the intention of the table in the first place, to try and highlight modules that were failing badly when they were uploaded, and in that respect it's a success. But it would have been much nicer to have only seen modules uploaded, long before CPANTS appeared, in the list.

I got a few responses to my mailout to testers for mapping their email to their real name, but it would seem that despite my efforts we have tipped over the 1000 testers edge again. Expect a bit of investigation work to bring that back under for next month ;)

Not the intent at all

Alias on 2007-04-04T15:08:23

My intent wasn't to find modules failing everywhere, because often they are going to be modules of little consequence.

I preferred the ranking based on absolute FAIL totals because it highlights old and important modules like, say, Template Toolkit that are failing a lot. My assumption is that the more-important the module, the more times it will be tested, and thus 10 fails out of 10 testers on a little unimportant module is equal to 10 out of 150 testers on something critical like File::Spec.

The way you have it now means that if ANYONE can ever make a module work, even if it is broken for the vast majority of cases, the failure is hidden.

Re:Not the intent at all

barbie on 2007-04-04T19:20:31

The way you have it now ...

Is that singular or plural? If the former, there is the absolute failure counts as well ;)

Re:Not the intent at all

Alias on 2007-04-04T21:30:42
Ah, missed that. Goodo