I just released Test::Harness 2.57_06. I removed failing test percentages from the output. It's always seemed silly to have shades of grey in what is a black/white issue: Did the tests pass or fail?
Re: Tests are a binary issue
Adrian on 2006-04-23T09:13:36
I don't like the change myself. I'm bright enough to figure out that anything less than 100% pass is bad when developing.
When using other peoples test suites seeing, for example, 99% ok tells me something very different from seeing 3% ok. For me the difference between "nearly there apart form this bit of functionality that I don't care about" and "completely f**ked" is useful. Yes I can figure it out from the test/pass numbers - but the percentage gives me a handy overview. Math is hard!
:-)
Not something I feel
/that/ strongly about - but I don't see the utility of the change myself (beyond code simplification in T::H).
Re: Tests are a binary issue
dws on 2006-04-23T16:49:40
An implied question when I do a test run is "Did I screw up something fundamental, or just some edge case?" Numbers help me get that answer.
Re: Tests are a binary issue
petdance on 2006-04-23T16:56:55
You get the numbers. You just don't get the percentages.
There's no difference between blowing 3 tests out of 100, or 3 out of 10,000. So 97% is no worse than 99.97%.
Re: Tests are a binary issue
dws on 2006-04-23T20:14:05
As an alternative, how about replacing the percentages with some pithy text?
"You missed one."
"A pair slipped by."
"You've got a few problems."
"Uh, oh. Not good."
"Hands up, and back away from that keyboard."
Re: Tests are a binary issue
Aristotle on 2006-04-23T23:29:05
Maybe if Coy is loaded?
Computers are binary, humans aren't
brian_d_foy on 2006-04-23T21:30:14
I use the percentages to track progress, and it's something I can point customers to.
Was there some other reason for this change?
Re:Computers are binary, humans aren't
petdance on 2006-04-24T00:40:05
Other than removing extra code, and not wanting to support the "correctness" of that percentage, no. I had a ticket in RT where 99.994% rendered as 100%, which was misleading.
But mostly it was because it just doesn't make sense to talk about tests in a way other than pass/fail.
The whole end-of-the-run reporting system is on my short list to overhaul anyway.
Makes sense to me
grantm on 2006-06-07T21:32:12
Given that you are still reporting the number of failed tests, not displaying the fairly meaningless percentage is no loss at all.
