Where The Hell Is Test::Builder2?

schwern on 2010-06-27T19:16:09

My progress and communication about the Test::Builder2 grant has been nothing short of appalling. There is a sort of herky-jerky progress where I figure out a design problem, push the code forward, then remember a use-case that throws a wrench in the whole design and the whole thing comes to a screeching halt again.

At the QA hackathon we elegantly solved the problem of things like die-on-fail and Test::NoWarnings but then ran afoul of things like Test::Warn and Test::Exception which runs tests inside of tests but those aren't actually part of the test stack.

Confused? I'll post more about it another time. Point is, TB2 continues to move forward, its just that there's long periods of rumination between sprints of development. And I get distracted by other projects. At this rate I'll be collecting Social Security before I collect the second half of the grant. I really want TB2 to happen, but something decisive has to be done. I work best with hard deadlines, so the plan is to clear a month for working mostly on TB2. A lot of the wibbling is trying to come up with the most elegant solution, but I usually have a less than elegant way to solve it and move forward. If its between an elegant TB2 that doesn't exist and a less elegant TB2 that does, well, go with the one that exists. With the way my schedule is looking, that will probably be mid-August to mid-September. If that doesn't produce an alpha, then I'll kill the grant.

That doesn't mean TPF gets nothing for your money. Chunks of TB2 can be harvested to improve TB1. Specifically, the TB2 formatting and history objects. The TB2 formatter makes the guts of TB1 cleaner, and it also allows it to produce something other than TAP. Used together, history and formatter allows non-Test::Builder based test frameworks to work together with Test::Builder providing even more flexibility. This is already done in the TB2 branch.

Looking at the grant deliverables, most of it is done:

* Split up global shared Test resources into individual objects
    * The test counter
    * The output filehandles
    * The plan

* Allow hooks for global beginning and end of test functions. * Ensure multiple hooks "stack" * die on fail * debug on fail

* Hooks for global beginning and end of test actions * Example: A safer Test::NoWarnings * Example: Don't cleanup temp files on failure so they can be debugged

* Allow for test output other than TAP

* Allow another Test::Builder-like module to work in the same process as Test::Builder (for example, sharing the counter).

* Rewrite Test::Builder in terms of Test::Builder2.


Here's what's not complete:

* Split up localizable behaviors into objects

* Allow individual test modules to locally override Test::Builder2 behaviors

* Allow test modules to globally override Test::Builder2 behaviors * How the plan works


Since I'm not writing to the letter of the law, there's more than that to be done before release, but the project does move.