A TAP Parser In 40 Lines of Code

Ovid on 2008-04-10T10:09:36

This is a reasonable TAP parser in 40 lines of code:

sub tap_failed {
    my $tap      = shift;
    my $plan_re  = qr/1\.\.(\d+)/;
    my $test_re  = qr/(?:not )?ok/;
    my @failed;
    my $core_tap = '';
    foreach ( split "\n" => $tap ) {
        if ( /^not ok/ ) {
            # TODO tests are not failures
            push @failed => $_ unless
                m/^ ( [^\\\#]* (?: \\. [^\\\#]* )* )
                 \# \s* TODO \b \s* (.*) $/ix
        }
        next unless /^(?:$plan_re|$test_re)/;
        $core_tap .= "$_\n";
    }
    my $plan;
    if ( $core_tap =~ /^$plan_re/ ) {
        $plan = $1;
    }
    elsif ( $core_tap =~ /$plan_re$/ ) {
        $plan = $1;
    }
    return 'No plan found' unless defined $plan;

    if ( @failed ) {
        my $failed = @failed;
        return "Failed $failed out of $plan tests";
    }

    my $plans_found = 0;
    $plans_found++ while $core_tap =~ /^$plan_re/gm;
    return '$plans_found plans found'
      if $plans_found > 1;

    my $tests = 0;
    $tests++ while $core_tap =~ /^$test_re/gm;
    return "Planned $plan tests and found $tests tests"
      if $tests != $plan;

    return;
}

OK, it's not really a parser. It only tells you if the tests failed or passed. There's tons of information which it discards and it does virtually no validation of the TAP structure, but might solve the nested TAP problem.

---

Almost, but...

mpeters on 2008-04-10T15:47:11

What about the return code of the stream? That's YAML, so you'd need a YAML parser. Then there's the whole discussion about the context: what if the status of the test is changed based on the context in which it was run? What if TAP changes? Say you have a TAP emitter which emits V14 and then it spawns another process that emits V15 and your V14 emitter is trying to capture that and nest it. Will the pass/fail rules you have for V14 work for V15?

I'm still not convinced that having a TAP parser, even a really simple one, is a good idea for a TAP producer.

Returns Codes

Ovid on 2008-04-10T18:34:03

Returns codes can be a legitimate concern, but in many cases these are unavailable anyway. Also, context does not change whether or not a test succeeded or failed, it changes the interpretation of whether it succeeded or failed. However, if something fails, it's better to default to failure in the absence of more information.

As for the v14/v15 issue, if we guarantee that TAP core doesn't change, we're safe. This entire work is designed to eliminate the need to break backwards-compatibility. I think we were far too rushed to make such a decision lightly.