Running Tests In Multiple Languages

Ovid on 2008-12-08T11:23:14

Recently on the TAP development list, the question of running tests in multiple languages came up. The example was PHP and Perl, but I'll show Ruby and Perl as I don't have PHP installed on this machine.

First, create the following Ruby program (adjusting the shebang line to taste):

#!/usr/bin/env ruby -w

'hello?'

puts '1..2';
puts 'ok 1 - ruby 1';
puts 'ok 2 - ruby 2';

Now run that with the prove utility which ships with Test::Harness.

prove -v test.rb
test.rb ..
1..2
ok 1 - ruby 1
ok 2 - ruby 2
test.rb:3: warning: unused literal ignored
ok
All tests successful.
Files=1, Tests=2,  0 wallclock secs ( 0.03 usr  0.01 sys +  0.00 cusr  0.01 csys =  0.05 CPU)
Result: PASS

Note that we're really running Ruby code, even with a Ruby error message. How does that work? You can run perl test.rb and you'll see that when Perl encounters the shebang line, it's smart enough to DWIM! This means that if you have shebang lines, prove will DWIM also. So, let's assume that our Ruby tests end in .rbt, our PHP tests end in .phpt and Perl tests end in .t:

alias myprove='find t/ -name '*.t' -or '*.phpt' -or '*.rbt' | prove -

Then just run myprove and you can run test suites written in multiple languages. (You can do this with tests without shebang lines, but that's another post).

Note that you can't just use xargs here. I had originally suggested that, but Andy Lester pointed out to me that this would invoke prove multiple times. A colleague of mine explained to me that xargs will cause this behavior if the generated list is too large -- as often happens with a find command.


What about a ruby TAP emitter?

jeremiah on 2008-12-08T12:11:44

Can you recommend a ruby TAP emitter?

I have had not-so-great luck with what is on the TAP wiki as far as TAP producers are concerned and I would like to create a collection of tools that spit out TAP in a variety of languages.

Re:What about a ruby TAP emitter?

rjbs on 2008-12-08T12:53:50

There's the awful, terrible, nearly useless one embedded in http://github.com/rjbs/rx/tree/master/ruby/rx-test.rb

so... I'd love to know if there's a better one, too.

Re:What about a ruby TAP emitter?

Ovid on 2008-12-08T14:27:39

I don't know. From what I've read, the Ruby people seem to have their own test tools that they're quite happy with. If you can coerce any of these tools to spit out TAP, that would be awesome.

Seems okay to me . . .

jeremiah on 2008-12-09T00:31:08

What is so wrong with that code? I am no ruby expert but it seems okay to me. I will try to use that to see if I can get it to work.