On testing modules

gambuzino on 2005-05-19T19:28:04

I've been getting some good progress on the Flickr-Tools distro, mainly on the lower-level modules (Flickr::API::Photos, Flickr::API::People, etc). At this point I can already rewrite the pet project which got me into writing this group of Perl classes in the first place, using only these classes. Admitedly this is not much to boast about, but it feels nice anyway... :-).

On the way I've met some dead-ends and had to track back a few times, but nothing too alarming.

Only now I've come across a real stumper.

While developing the modules I've taken the "build your tests before writing any code" approach which is all fine and dandy while I'm working on it, but I just don't know how I'm going to release the modules with the tests as they are now.

The problem is that testing anything useful at all requires not only the usage of a Flickr API key but also some rather precise information on a person's info, a few test pictures, photosets, and so on.

Of course I'm using my own information for the tests, but I'm not really happy with releasing a module into the wild with all that information on it!

I know I could just capture the result of the original Flickr API response for each instance I needed it, change some values around and test my modules against that result but that is just evading the problem and a whole lot of extra work.

I just don't know how I'm going to tackle this one...

(This entry is cross-posted from my personal weblog at )


Ask in perl.qa

clscott on 2005-05-19T20:45:23

Ask the perl QA mailing list http://qa.perl.org

Re:Ask in perl.qa

gambuzino on 2005-05-20T10:42:56

Hum... didn't think of that one, thanks for the tip!

dummy acccount

jmm on 2005-05-19T21:34:27

Ask the Flickr people for permission to create an account for "Random J. Hacker" or "A. Test Case" or some such equally nonexistant peron, with defined info that will never be changed (or useful for identity theft purposes).

Re:dummy acccount

gambuzino on 2005-05-20T10:46:23

Yep, this will actually be plan B.

Right now I've got some good leads on a way which doesn't actually require me to connect to Flickr (using Mock objects) and I'll give it a try as it would allow the testing to go through even without Internet connectivity.

But if this fails your suggestion (or, if it fails just creating a free test user with some stock photos in there, like pfig suggested on my blog) will be the way to go.

Re:dummy acccount

Aristotle on 2005-05-21T08:33:13

Of course, using mock objects means your module will install flawlessly for users but then promptly bomb out on them, should the Flickr API ever change. I’d ask Flickr if they don’t already provide an account for testing. I remember that dyndns.org provided one for their proto-webservice years ago – basically, internet-centuries in the past. If Flickr are worth anything they’ll have anticipated this.

Re:dummy acccount

gambuzino on 2005-05-21T17:00:45

That's a point I've considered myself, but then I don't think this sort of testing is supposed to catch that.

As far as I know testing is supposed to ensure that given a "perfect" response from Flickr the instalation works for the user. In other words the tests aim at is ensuring the user has the environment correctly setup in order to be able to use the modules (and, of course, that I didn't screw up i nsome way that breaks the modules on their setup).

Changes in the API will always break the modules and I must deal with them anyway, but since I won't be running "make test" every day to catch that possibility I don't think that is much of an issue...

Does this make sense?