Automated Javascript tests with Test.Simple + WWW::Selenium

schwern on 2009-06-06T23:01:02

I've been doing acceptance level QA at my $job lately which means a lot of clicking around in browsers and a lot of writing Selenium tests. Really my job is to reduce the amount of manual testing which needs to be done and automate as much as possible.

I was talking with Zack who said he hates Selenium. What he really meant was he hates testing at the browser level. Its so finicky to write Selenium regression tests that won't break later because the layout changed. He'd rather unit test his Javascript. I pointed out Test.Simple as a solution. Trouble is, that runs in a browser and someone still has to look at it. That's not very useful for automated tests.

So I began wondering... what would it take to pipe the Test.Simple TAP into TAP::Harness? Ideally I want to run the Javascript with a real DOM in a real browser, not some tinker toy simulation. Can I get Firefox to pipe its page rendering to STDOUT? I asked David Wheeler if he knew of anything and pointed me at some unfinished things like JSAN::Prove which requires a bunch of complicated setup and dependencies which I tl;dr'd.

Maybe there's some module on CPAN which can pipe from Firefox. I searched for "firefox" and what comes up but WWW::Selenium. OF COURSE! I can use WWW::Selenium to talk to Selenium Remote Control and get the output of a web page!

    require WWW::Selenium;

    # Assumes you have a selenium server running locally on 4444
    my $sel = WWW::Selenium->new(
        host => "localhost",
        post => 4444,
        browser => "*firefox",
        browser_url => "file://nothing"
    );

    $sel->start;
    $sel->open('http://isperldeadyet.com');
    print $sel->get_body_text();

Then it's a simple matter of hooking this into TAP::Harness.

#!/usr/bin/perl -w

use TAP::Harness;

my $harness = TAP::Harness->new({
    exec => sub {
        my( $harness, $file ) = @_;

        # Let Perl programs run normally
        return undef unless $file =~ m{\.(js|html)$};

        require WWW::Selenium;
        my $sel = WWW::Selenium->new(
            host => "localhost",
            post => 4444,
            browser => "*firefox",
            browser_url => "file://nothing"
        );

        require File::Spec;
        my $url = "file://" . File::Spec->rel2abs($file);
        $sel->start;
        $sel->open($url);

        # Get whatever's inside 
        return $sel->get_text(q{//pre[@id="TAP"]});
    },
    verbosity => 1
});


Then write up a little HTML wrapper to run a basic Test.Simple test.
(Note that t/lib contains the Test.Simple libraries)



  
    
    
    



  





And run it.

$ perl -w javascript_harness.plx tap.html
tap.html .. ok   
All tests successful.
Files=1, Tests=1,  8 wallclock secs ( 0.19 usr +  0.03 sys =  0.22 CPU)
Result: PASS


Its slow, but it's awesome!  It gives me something to stick into an
automated smoke server.  Developers need a faster turn around time, so
they can quickly and individually unit test it directly in their browser using
Test.Harness.Browser... but that's another show.

But wait, there's more!

That HTML wrapper is icky.  Wouldn't it be better if one could just
test Javascript directly?  Why yes!  How about we generate the wrapper
for .js files?

#!/usr/bin/perl -w

use autodie;
use TAP::Harness;

my $harness = TAP::Harness->new({
    exec => sub {
        my( $harness, $file ) = @_;

        my($type) = $file =~ m{\.(js|html)$};
        return unless $type;  # run Perl normally

        require File::Spec;
        require WWW::Selenium;
        my $sel = WWW::Selenium->new(
            host => "localhost",
            post => 4444,
            browser => "*firefox",
            browser_url => "file://nothing"
        );

        my $url = $type eq 'js'   ? _testify_javascript($file) :
                  $type eq 'html' ? File::Spec->rel2abs($file) :
                                    die "Unknown type $type";

        $sel->start;
        $sel->open($url);
        return $sel->get_text(q{//pre[@id="TAP"]});
    },
});

$harness->runtests(@ARGV);


# Turn .js files into .html with the Test.Simple libraries loaded.
sub _testify_javascript {
    my $file = shift;

    open my $fh, "<", $file;
    my $javascript = join "", <$fh>;

    use Cwd;
    use File::Temp 'tempfile';

    my $cwd = cwd;

    my($tmpfh, $tmpfile) = tempfile( CLEANUP => 1 );
    print $tmpfh <<"END_OF_HTML";


  
    
    
    







END_OF_HTML

    return "file://$tmpfile";
}


Which eliminates all the scaffolding from the Javascript test file.

plan({ tests: 1 });
ok( 1, "this is a test from Javascript" );


And the end result is I can run Javascript and Perl tests together from the command line.

$ perl javascript_harness.plx t/*.js t/*.t
t/tap.js .. ok   
t/perl.t .. ok   
All tests successful.
Files=2, Tests=2,  9 wallclock secs ( 0.19 usr  0.04 sys +  0.03 cusr  0.01 csys =  0.27 CPU)
Result: PASS


UPDATE:  As threatened, its on github.







Cool
grink on 2009-06-07T02:20:55
Have not dug into the nitty gritty of your post yet, but wow, looks awesome



feels good man.
drowsy on 2009-06-07T13:51:12
it looks very helpful, thank you xwrn!



Thanks
markjugg on 2009-06-08T01:59:14
Looks like a useful breakthrough. Bookmarked.

Thanks!

   Mark