One of the features I'm working on for TAPx::Parser is the ability to test anything. Admittedly, this is a big attempt, but here's what I'm thinking. Imagine the following program:
#!/usr/bin/perl
use Test::HTML::Lint tests => 1;
my $file = shift;
open my $fh, '<', $file
or die "Cannot open ($file) for reading: $!";
my $html = do { local $/; <$fh> };
html_ok( $html, "$file has valid HTML" );
Now that looks weird. Why would I want arguments to a test script? Well, name it test_html and you can do this ('extension' is deliberately misnamed):
runtests --recurse --exec 'test_hml %s' --xtension 'html?' www/
You've now turned all HTML files in your www/ directory into tests!
--exec is expected to take a sprintf format which will be opened with this:
my $fh = gensym;
my $command = sprintf $exec, $arg;
if ( open $fh, "$command |" ) {
return TAPx::Parser::Iterator->new($fh);
}
else {
...
}
So long as whatever you pass into --exec outputs TAP, you can test anything you want. In fact, it doesn't even have to be filenames. You could pass a list of urls to runtests and have your testing utility fetch them and run tests on them.
What I'm trying to figure out is how to easily handle a test suite with test programs written in Perl, C, etc. My current plans only allow one type of --exec at a time. Thoughts?
Update: I might just make the %s optional in the format and tack it on to the end if it's not in the string. I'd also disallow any other % sign in the format since that would screw up the sprintf.
You might not want to use sprintf and the string form of pipe open. Think what happens when you get a file name containing spaces, or asterisks, or pipe characters, or anything else that's special to the shell (yes, I actually had such names: one of my applications used TT2 to generate documents, and the file names themselves were expanded by TT2, so I had names like [% customer.name | filename %] - [% orders.last_number | format('%04d') %].txt )
I'd suggest doing what find's exec option does: gobble up all parameters up to a literal;, and substitute the filename in place of a literal {}.
Re:URLs?
Ovid on 2007-01-04T09:45:00
This feature is not yet ready but I'm already getting email asking for it so I need to see if I can get this out the door by this weekend at the latest. My biggest obstacle, actually, isn't getting this feature to work (it's fairly simple), but getting TAPx::Harness output as similar as possible to Test::Harness output. I don't want to suprise users too much by making the output radically different, but I also have so much more information I can present, if they users want it (such as which tests unexpectedly succeeded), that the output has to be different in some ways.
On the off chance you were asking a different question, here's a more complete description of the rationale and implementation.
In Schwern's original plan for this, he wanted to have a bunch of separate "source" modules:
- TAP::Harness::Source::String
- TAP::Harness::Source::PerlDotT
- TAP::Harness::Source::HTMLValidator
- TAP::Harness::Source::HTTP
- TAP::Harness::Source::Executable
- ... and so on
You can go back to that link and see what he was thinking, but I've realized that the bottleneck in that approach is that most end users are going to wait for someone else to write a TAP::Harness::Source::... module for their particular needs rather than write it themselves. Or such a source module will already have been written but it's features will be too limited. My approach allows you to write a simple test program which takes an input and figures out how to test it. For example:
#!/usr/bin/perl
use Test::More tests => 4;
use Test WWW::Mechanize;
my $mech = Test::WWW::Mechanize->new;
my $url = shift;
$mech->get_ok(
$url,
"We should be able to fetch ($url)"
);
$mech->title_like(
qr/^Login to control panel for/,
"... and page should be a customer control panel login page"
);
$mech->content_like(
qr/Copyright 2007 by Acme Corp/,
'... and it should have our copyright'
);
$mech->page_links_ok("... and links from this page should be good");Then you just use any of a variety of methods to pass a list of URLs to it. One of the simplest, if you name the above program test_control_panel_logins:
runtests 'test_control_panel_logins' < list_of_control_panel_urls.txtThus, as long as you can figure out how to write a test for something, you could use runtests to test anything.