in my last journal entry, i described the file and directory naming for the official Perl 6 specification tests. this will make it easier for Perl 6 implementors to find tests related to the pieces of the compiler under construction, and will encourage what i like to call "specification-driven development" -- write the spec first, then the tests, then the implementation.
here i'll address a critical bit of infrastructure necessary to write a test suite destined to be run by multiple implementations. since the various implementations of Perl 6 will be sharing the same test suite, we can't mark tests that are todo or should be skipped as done in a typical CPAN module. each implementation will need to classify its own set of failing tests that should be skipped or marked todo. here's a list of our guiding principles:
~ the skip/todo classification should be as close to the test as possible.
we'd like to avoid a situation where there's one big file that has a list
of skip/todo. this is fragile, because as tests are modified, maintained,
and refactored, it will be quite difficult to keep this solution working--
especially across multiple implementations.
~ compilers should recognize only the directives that impact them.
we need to be sure that kp6 doesn't run directives intended for pugs, etc.
~ the directives may be used for purposes outside the realm of testing.
although testing Perl 6 is the driving requirement for this solution,
and we have no other plans for these compiler directives at this time,
we recognize that it's likely that they'll be useful elsewhere.
therefore, we're designing it in from the start.
various bits of syntax were offered, but we've settled on the following:
~ compiler directives will look like a comment to all other compilers
in order to achieve this, we've settled on a prefix of '#'.
~ compiler directives are conditional by design (they're complier-dependent)
we append a '?' to the prefix to better reflect their conditional nature.
~ a compiler token must be specified
compilers need a token to know which directives are intended specifically
for them. we've chosen the compiler name.
(e.g. 'perl6', 'kp6', or 'pugs')
~ the marker should be visually offset from the body
purely for humans to make it a little easier to see, we've put a colon (':')
after the compiler name.
~ any Perl 6 term can follow a compiler directive
we don't have any need for multi-line directives at the moment,
so we haven't invented any syntax to support it.
putting it all together, the compiler directives look like this, shown inside a simple test file:
use Test;
plan 3;
#?perl6: todo( 'not yet implemented', 1 );
#?pugs: todo( 'feature', 3 );
#?kp6: skip( 'does not parse', 2 );
ok( 0, 'foo' );
ok( 1, 'bar' );
is( 1, 1, 'baz' );
given those directives, perl6 will mark the first test todo, pugs will mark all three tests todo (which should result in two bonus todo tests passing,) and kp6 will skip the first two tests. running these tests will result in all tests passing for each of these three implementations.
i've implemented this compiler directive syntax in the perl6 compiler, as well as these forms of 'todo()' and 'skip()'. i've included links to those commit diffs below.
i'll follow up soon with more information about the forms of 'todo()' and 'skip()' shown above, as well as two new test directives that will take advantage of this compiler syntax to mark blocks of tests.
related links:
Perl 6 Test Suite Refactor - file and directory naming:
http://use.perl.org/user/particle/journal/35366>
perl6 grammar:
http://svn.perl.org/parrot/trunk/languages/perl6/src/parser/grammar.pg>
[perl6]: support compiler directives with syntax '#?perl6:
[perl6]: add todo() to Test.pm
http://perlsix.org/svn/parrot/revision/?rev=24751>
