All the Perl that's Practical to Extract and Report

New testing tool in Test::Harness 2.32

petdance on 2003-11-07T20:51:39

I've uploaded the new Test::Harness, version 2.32. (Also available at the PAUSE FTP site if it's not updated yet on search.cpan.org)

Test::Harness now comes with a command-line utility called prove that runs tests against the harness. Why have a separate utility? What's wrong with "make test"?

prove has a number of advantages over make test when doing development.

  • prove is designed as a development tool
    • Perl users typically run the test harness through a makefile via make test. That's fine for module distributions, but it's suboptimal for a test/code/debug development cycle.

  • prove is granular
    • prove lets your run against only the files you want to check. Running prove t/live/ t/master.t checks every *.t in t/live, plus t/master.t.

  • prove has an easy verbose mode
    • To get full test program output from make test, you must set HARNESS_VERBOSE in the environment. prove has a -v option.

  • prove can run under taint mode
    • prove's -T runs your tests under perl -T.

  • prove can shuffle tests
    • You can use prove's --shuffle option to try to excite problems that don't show up when tests are run in the same order every time.

  • Not everything is a module
    • More and more users are using Perl's testing tools outside the context of a module distribution, and may not even use a makefile at all.

I welcome your comments and suggestions about prove, and I hope you find it as useful as I have over the past couple years of development.

Cool!

samtregar on 2003-11-07T22:42:28

Only question, why prove? I might have expected runtests or testrun or some such. Too bad test is taken.

-sam

Re:Cool!

petdance on 2003-11-07T23:26:34

I knew I shoulda put a section called "Why prove?" in the docs. :-)

First, test was right out, even if it wasn't already taken. It's one of those words that when I see it as a file I assume is garbage, like temp or foo. runtests is 8 characters, and it's two words. I wanted a single word. The program started as smoke at my day job, but smoke already has a meaning to Perl, or at least to the people who run the builds.

So I had to have a single word, and I convened an impromptu brainstorming session in AIM one day. I don't remember who was in it: Piers Cawley, David Hand, Joe McMahon and Chris Nandor, at least. We threw ideas around for about half an hour until someone suggest "prove". Here's what Merriam-Webster says about prove:

1 archaic : to learn or find out by experience
2 a : to test the truth, validity, or genuineness of <the exception proves the rule> <prove a will at probate>
b : to test the worth or quality of; specifically : to compare against a standard -- sometimes used with up or out
c : to check the correctness of (as an arithmetic result)
3 a : to establish the existence, truth, or validity of (as by evidence or logic) <prove a theorem> <the charges were never proved in court>
b : to demonstrate as having a particular quality or worth <the vaccine has been proven effective after years of tests> <proved herself a great actress>
4 : to show (oneself) to be worthy or capable <eager to prove myself in the new job>
"To test the validity of", "to check the correctness of", "to show to be worth or capable": Sounds like what we're trying to do, right? There's also the idea of a "proving grounds" that ties in with those.

And it's only 5 characters and easy to type, which is key.

Re:Cool!

koschei on 2003-11-08T08:19:58

Heh. taken would be a great name.

"Why is it called taken?"
"Because test is taken."

Allow some pre-test code?

cbrandtbuffalo on 2003-11-09T19:32:19

We often need to pass environment variables to our tests to simulate a particular run environment. We also tend to have logic wrapped around these variables to set parameters for sandbox testing, QA testing, etc.



Would it be possible to pass something into prove to make it run some pre-test code? If given a certain flag, maybe have it look for a default config-type file in the current directory and require it before running?



Just a thought...thanks for the new tool!

'prove' is useful

markjugg on 2003-11-12T17:53:40

I've already found this useful. using "make test", it was hard to remember the syntax to run just some particular tests and be verbose about something. This made it easy to re-test a particular script for DBD::Pg:

prove -v -Iblib/lib -Iblib/arch t/01setup.t t/15table_attrs.t t/99cleanup.t

Re:'prove' is useful

petdance on 2003-11-18T19:07:51

And this is now easier as
prove -v -b t/01setup.t t/15table_attrs.t t/99cleanup.t