Making It Easier For Users to Report Test Failures

chromatic on 2005-05-30T21:23:58

Perl's module tools make it easy to bundle, distribute, and install software, most of the time. Other tools make it fairly easy to test software too. Of course, the tools for end-users lag somewhat behind the tools for developers, especially in places where developers are happy with their tools.

If you're a Perl-savvy developer and tests fail in a module you're installing, you know what your options are. Consider what a non-developer could do, though. Yet developers who ship their tests for end-users to run rely on receiving feedback about failures so that they can fix the code, the tests, the assumptions, or whatever's not right.

I've just released Module::Build::TestReporter which runs the tests as usual, hijacks their output, keeps a log of any failures and their diagnostics, and tells users what to do to report any failures to the developers. If you think this will solve a problem for you, give it a whirl. (I'd love to have feedback before I release it to the CPAN in a week or so.)


M::B, tests & CPANPLUS

barbie on 2005-06-01T12:54:06

One of the tough tasks for CPANPLUS at the moment is getting a reasonable hold of the test results from M::B. Because it works through the API, and not via a command call, results are lost, unless wrapped in an eval. This doesn't work if the tests passed and you still want to see the results. Is this going to enable successful results to be reported in verbose mode?

I haven't had time to look at your module, but I certainly like the idea of a better test capture and reporting mechanism for M::B. CPANPLUS has a wrapper around EU::MM to get its results, but it has been a little harder to get at with M::B.

Re:M::B, tests & CPANPLUS

chromatic on 2005-06-01T17:06:19

M::B::TR doesn't currently capture success reports, but it uses Test::Harness::Straps, so reporting successes and failures is fairly easy.

Looks good

markjugg on 2005-06-01T15:29:55

I agree the problem space seems worth addressing, and the docs and code looked reasonable to me at first pass.