Who tests the testers?

2shortplanks on 2002-02-19T00:11:09

Well, this weekend has been pretty busy perl hacking.

Mostly I've been working on a module called - for now - Test::GD. The idea behind the module is that it does what Test::More does for strings, but for GD images. With it you can write a selections of tests to say things like "this image should be 200x200 in size" or "the pixel at (25,12) should be red". And the module will either print out "ok 1 - image size" if it's okay or "not ok 1 - image size" followed by some debug information if it's not, in Test::Harness compatible way.

This is of course all based on Test::Builder. And I'm testing that with my Test::Builder::Test module. You may recal me chatting about this a bit in previous journal entries - it's a module that allows you to test tests by declaring what they're going to produce in advance and then running the tests and comparing the exected and actual output. Well, there's nothing like using a module in anger to shake it up a bit and realise what features you needed to add!

The first function I added was a function that determines what line number a statement is on, rather than hard coding in the line numbers into the expected debug output of the test. Now rather than saying

  test_err("#     Failed test ($0 at line 12)";

I can say

  test_err("#     Failed test ($0 at line ".line_num(+1).")");

Of course, I got really bored of this real quick for the standard test message above that occurs on every test that fails and I added a quick method that means I can just do

  test_fail(+1);

For those simple cases. Other debug information has to be output in the form

   test_err("#    You forgot to change your socks");

Which was also tiring, so I added another convinience method to do this

   test_diag("You forgot to change your socks");

Instead, which is a lot simplier. So now I have a lot more convinent module to write tests with. Yay!

The main problem with testing tests is the quality of the debug information that you get out of a module when the test fails. Because you're quoting the diagnostic text that you expected and what diagnostic text you actually got back from the test you were testing in the diagnostic text that you're outputting, it's really hard to tell what diagnostic text is what.

I played with many potential solutions, but in the end I decided to take advantage of the Term::ANSIColor module to add colour to my diagnostics if you want them. So now the code can (if you enable it) print out the matching diagnostic text in green and differing diagnostic text in red. This is a lot clearer and makes the whole process a lot quicker.

But man, was that hard to test. I have tests that check the tests that test Test::Builder's test function's output (which in turn is a module that I'm using to write tests to test the output of Test::GD's tests, which in turn again will hopefully used to write tests that test another of my pet projects GD::TextImage.) Let's just say by the end of the process I was very confused - but at least I got an "All tests successful."...

This is all available from: http://2shortplanks.com/temp/tbt/- please download it and have a play.

Anyway...

blech has a concept of brain farts. This - for want of a better description - is an idea that you come up with that might be cool, but you never know if you're going to implement it or not. It's just "at the conceptual stage"

I was thinking about GD::Test and the is() function. This enables you to check that pixels are of the correct colour or not. I got to thinking what if you want to check if a pixel is yellow or blue. There's currently no way of doing that. If I was doing a test on a string using Test::More I'd use like() and a regular expression for both options. Hmm.

Regular expressions for images.

Now there's a concept. I'm not sure how these would exactly work. You could have them function left to right, row by row. This would make them analogous to regexes for strings. Or you could allow them to move to any adjacent pixel. This would make them more like xpath, where you can move in many directions (and even more NFA than DFA.)

The implementation aspect boggles the mind. Could I map an image to a string and use the normal regex engine? Would I want to?

That's the good thing about brainfarts....I don't have to worry about that just yet ;-)


Image regex

kasei on 2002-02-19T01:09:17

Hahaha... if the regex allowed moving to any adjacent pixel, you could write a regex to solve a black-and-white maze image ;)



So going from the red starting pixel, to the green ending pixel travelling only on the white paths:

print "yay" if ($image =~ /\N{red}\N{white}*\N{green}/);



That would totally rule! I don't think mapping an image to a string would quite be feasible if you allowed adjacent pixels though... unless the string was lazily generated or something... wow - crazy thought!

.g

Re:Image regex

drhyde on 2002-02-20T11:25:37

The idea of being able to move to any adjacent pixel reminds me of Befunge. Don't go there!

Birthday bots

sphyg on 2002-02-19T01:33:25

At my sixth form college, we had a swipe card registration system which beeped at you on your birthday. I don't recall it beeping 'Happy Birthday' though ;-)