I've been working this week to add support YAML-based diagnostic syntax to TAP::Parser. Those changes haven't made it to CPAN yet but we'll probably make a release in the next few days. In the mean time you can find the bleeding edge version in the subversion repo.
Now TAP producers can elect to add machine readable diagnostics to test failures like this:
TAP version 13
plan 1..5
ok 1
ok 2
not ok 3
---
severity: fail
message: "Array unsorted"
data:
got:
- 1
- 3
- 2
expected:
- 1
- 2
- 3
...
ok 4
ok 5
The harness doesn't currently actually do anything with the diagnostic information but it does expose an interface that allows display plugins to be written. With the right plugin the test harness becomes responsible for displaying the detail of what went wrong. Currently that's something test scripts must do themselves, calling diag() to display a description of the problem. Because diag() outputs to STDERR the diagnostics can get out of sync with the test results.
Look out for more progress on display plugins and a practical end-to-end example of a test that emits structured diagnostics sometime next week.