During YAPC I observed someone[1] writing tests, putting
use Test::More tests => 1000;
at the top of the
.t file, and I realized -- I do that too! All the time.
There is a simple reason for this behaviour.
When you run
prove and there are fewer tests than in the test plan, it's fairly evident how many tests were run from the output. So it's easy to stick that number into the test plan once the tests work.
When you add more tests without updating the test count, the output says
"Looks like you planned 47 tests but ran 27 extra". There is no final count of tests run.
Being the lazy bugger I am I refuse to do that simple math operation in my head (because it's not simple, it makes my brane hurt and I risk getting it wrong). Instead I put 1000 at the top to get the answer served on a plate, which is what computers should do as much as possible[2].
TAPx::Parser's
runtests to the rescue. Using that test-runner instead of
prove doesn't only give me colored output, I also get a sane test count:
"You planned 47 tests but ran 74"
Actually I didn't realize
runtests did this wonderful thing. This whole journal entry was supposed to be a bit of whining that
prove didn't do the right thing damnit, and then when I ran some tests to give me the bad output it turned out to be fixed already!
So all this time I have been fiddling with
use Test::More tests => 1000;
for nothing. Doh!
Well, now I know. Problem solved. By whining.
[1] Edmund, how's it going with that smoke-performance-regression-over-time thingy?
[2] Serve me things. On a plate. Because no-one else does.
I just set no plan
phaylon on 2007-09-03T18:34:11
Since I usually run my testfiles with perl itself (rather than prove) while bootstrapping, I just skip the plan at first. WFM
:)
Re:I just set no plan
Aristotle on 2007-09-03T20:19:32
Perl QA wiki: Test::FAQ: How do I update the plan as I go?
Re:I just set no plan
jplindstrom on 2007-09-03T22:02:51
Hey, that's pretty clever!
But much more Done The Right Way(tm) using Test::Class, especially for testing large applications (rather than CPAN size modules) where elaborate setup code sometimes has embedded tests.
Re:I just set no plan
Aristotle on 2007-09-03T22:55:46
Yeah, Test::Class is nice that way, but it’s also a pretty big gun. The BEGIN block method is a nice way of making small test suites more tractable.
no math in head (use computer for that)
vim t/file.t
/tests<Enter>ww27Ctrl+a
:-D
Re:no math in head (use computer for that)
Aristotle on 2007-09-03T20:19:33
Perl QA wiki: Test::FAQ: How do I update the plan as I go?
Re:no math in head (use computer for that)
jrockway on 2007-09-04T10:31:16
I do something similar with emacs:
http://blog.jrock.us/articles/Increment%20test%20counter.pod
I like Test::Harness' reporting of the delta, since my script will accept a number to add to the plan, not the absolute number of tests.
However, that would be pretty easy to change if Test::Harness' output format changes.
Test::Count is a Better Idea
Shlomi Fish on 2007-09-04T09:00:35
See
my
article about Test::Count, for an introduction to the module, and why it is a better idea to keep track of the number of tests that are expected to run explicitly, by calculating them. Otherwise, it is possible that the number of tests that the script reports is different than the numbers that should run.
s/runtests/prove/
petdance on 2007-09-04T16:06:32
Please note that since TAPx::Parser is moving to be Test::Harness, runtests is now prove.
