I did it. I finally killed that damned threading test in Test::More. is_deeply_fail.t will no longer run unless AUTHOR_TESTING is set.
For those who don't know, that test would tickle intermittent threading bugs in certain vendor supplied versions of perl. The 5.8.6 OS X ships with and most Redhat derivatives are vulnerable, but it would only fail about one in a hundred times. I don't know what the problem is, I suspect it's from vendors pulling in bleadperl patches but I haven't confirmed. Nobody's put the time into it to find out.
Since there's nothing I can do about it, and it was just holding up CPAN installations, I turned the test off. Done.
Re:Why AUTHOR_TESTING?
Ovid on 2008-02-25T08:46:07
That's very much a mixed bag. On one hand, yes, you clearly would want it in those tests. On the other, it's not confidence inspiring to see a lot of test failures when you're evaluating a module. Of course, people say "yes, but you should check to see what causes those failures", but this is problematic. Many, many times I've received a failure report with absolutely no information about why something has failed. Or I get a failure report with spurious information, which is just as annoying.
If Schwern allows automated testing, not only does he have a good chance of getting bogus failure reports, he might find himself in the annoying position of getting into wasteful dialogues with some testers who aren't familiar with this issue and really, why should they be?
OK, that was all just spewing as I sorted out my thoughts. I guess what would be good is to have a special type of "no report" test. In other words, a test which can fail, sort of like a TODO, but which automated tests shouldn't report. I'm unsure of the mechanism here. Didn't someone on Perl QA mention something like this, or am I smoking crack?
Re:Why AUTHOR_TESTING?
schwern on 2008-03-02T10:28:38
Surely you want it in the CPAN Testers tests?They're not telling me anything I don't already know, there's nothing I can do about it and it's highly intermittent. Takes dozens or hundreds of runs before it shows up.
There's only so many times I can say "don't use Redhat's (or any distribution derived from Redhat) perl".
Re:Why AUTHOR_TESTING?
Aristotle on 2008-03-02T14:30:17
No, the point is, if there is an actual regression that these tests would catch, on a platform you don’t have, you won’t know about it if the CPAN Testers don’t run those tests.
How much of a concern that is depends on the tests in question.
Intermittent failures from CPAN Testers should be easy to ignore, in any case.
