The new TAPx::Parser is not on the CPAN yet, but if you remove the t/sample-tests/bailout from the sample test directory in the TAPx::Parser distribution and run prove across those tests, you get the following summary report:
Failed Test Stat Wstat Total Fail List of Failed ------------------------------------------------------------------------------- t/sample-tests/bignum 2 ?? ?? t/sample-tests/bignum_many 2 1999 3-100000 t/sample-tests/combined 10 2 3 10 t/sample-tests/die 1 256 ?? ?? ?? t/sample-tests/die_head_end 1 256 ?? ?? ?? t/sample-tests/die_last_minute 1 256 4 0 ?? t/sample-tests/duplicates 10 ?? ?? t/sample-tests/head_fail 4 1 2 t/sample-tests/inc_taint 1 256 1 1 1 t/sample-tests/no_nums 5 1 3 t/sample-tests/no_output ?? ?? ?? t/sample-tests/segfault 0 11 ?? ?? ?? t/sample-tests/simple_fail 5 2 2 5 t/sample-tests/switches 1 1 1 t/sample-tests/todo_misparse 1 1 1 t/sample-tests/too_many 4 1024 3 4 4-7 t/sample-tests/vms_nit 2 1 1 (2 subtests UNEXPECTEDLY SUCCEEDED), 2 tests and 3 subtests skipped. Failed 17/31 test scripts. -10/101 subtests failed. Files=31, Tests=101, 5 wallclock secs ( 0.34 cusr + 0.23 csys = 0.57 CPU) Failed 17/31 test programs. -10/101 subtests failed.
Note that there are lots of question marks. You also don't know which tests skipped or which tests unexpectedly succeeded. The number of tests which failed is also a bit weird. Here's the output from runtests:
Test Summary Report ------------------- t/sample-tests/bignum (Wstat: 0 Tests: 4 Failed: 2) Failed tests: 136211425-136211426 Errors encountered while parsing tap t/sample-tests/bignum_many (Wstat: 0 Tests: 11 Failed: 9) Failed tests: 99997-100005 Errors encountered while parsing tap t/sample-tests/combined (Wstat: 0 Tests: 10 Failed: 2) Failed tests: 3, 10 TODO passed: 9 Tests skipped: 7 t/sample-tests/die (Wstat: 256 Tests: 0 Failed: 0) Errors encountered while parsing tap t/sample-tests/die_head_end (Wstat: 256 Tests: 4 Failed: 0) Errors encountered while parsing tap t/sample-tests/duplicates (Wstat: 0 Tests: 11 Failed: 0) Errors encountered while parsing tap t/sample-tests/head_fail (Wstat: 0 Tests: 4 Failed: 1) Failed tests: 2 t/sample-tests/inc_taint (Wstat: 256 Tests: 1 Failed: 1) Failed tests: 1 t/sample-tests/no_nums (Wstat: 0 Tests: 5 Failed: 1) Failed tests: 3 t/sample-tests/no_output (Wstat: 0 Tests: 0 Failed: 0) Errors encountered while parsing tap t/sample-tests/out_of_order (Wstat: 0 Tests: 15 Failed: 0) Errors encountered while parsing tap t/sample-tests/simple_fail (Wstat: 0 Tests: 5 Failed: 2) Failed tests: 2, 5 t/sample-tests/skip (Wstat: 0 Tests: 5 Failed: 0) Tests skipped: 2 t/sample-tests/skip_nomsg (Wstat: 0 Tests: 1 Failed: 0) Tests skipped: 1 t/sample-tests/todo_inline (Wstat: 0 Tests: 3 Failed: 0) TODO passed: 2 t/sample-tests/todo_misparse (Wstat: 0 Tests: 1 Failed: 1) Failed tests: 1 t/sample-tests/too_many (Wstat: 1024 Tests: 7 Failed: 4) Failed tests: 4-7 Errors encountered while parsing tap t/sample-tests/vms_nit (Wstat: 0 Tests: 2 Failed: 1) Failed tests: 1 Files=31, Tests=122, 2 wallclock secs ( 1.07 cusr + 0.39 csys = 1.46 CPU)
It reports more tests as having been run because it can count them more accurately. It also has no problem figuring out which tests have been skipped, which have unexpectedly passed (these are passing 'TODO' tests) and if you want to see those parse errors in detail, you just pass it the '-p' switch.
It's still not perfect, but it's getting one heck of a lot closer. Mine's also more verbose, but I'm reporting a heck of a lot more information. If you don't like the report format, all you'll have to do is override the &TAPx::Harness::summary. Lots of other things can also be overridden, if need be.
