I posted about my WWW::Selenium + Test.Simple hack yesterday to enable automated Javascript unit testing. One of the problems was it was very slow. It had to start and kill a Firefox instance between each test which takes 8 seconds per test on my machine. Running 7 tests is a full minute.
Solution? Cache the selenium object! This will reuse the same Firefox session between tests so you only get slammed by the startup cost once. Now my 7 tests run in 8 seconds, the time to start up Firefox. That's awesome!
Will reusing the same Firefox process cause a problem? Unlikely. When I test web sites, with or without Selenium, I sure don't restart Firefox between checks. And neither will your users, so this is far more realistic. Web browsers are designed to isolate page requests from one another.
The prototype works. Future directions...
* Roll selenium-server into the distribution.
* Automate starting the selenium server.
* Add a config file...
* Which browser(s) to use?
* Which selenium server to use, or start its own?
* What file extensions to test with selenium?
* Rerun tests across multiple browsers
* Turn the HTML wrapper into a configurable template
* Make it play nice with prove.
* Turn it into something which can be used with --exec
* Turn it into something which can be put into .proverc
* Modularize it
* Figure out how to keep the Firefox process from appearing
* Or at least run backgrounded
Will reusing the same Firefox process cause a problem?
Your arguments in favor of reusing the process are very similar to my arguments in favor of running tests in a single process with Test::Aggregate. Even though it's worked very well, a few people raked me over the coals for this decision. I wonder if they'll give you the same level of grief
Re:Reusing a process
schwern on 2009-06-08T09:40:38
A web browser is designed to keep things strictly separated. Perl is, at best, a half-blind chaperonage in a French farce.
Test::Aggregate is the mod_perl of the testing world. If you take a bunch of things which were all designed to run in their own process once and then terminate and try to smash them together they're going to step all over each other and themselves. It requires careful recrafting, and restricting, of the formerly independent tests for them to all run together. Even then you can't be sure a failure wasn't due to some sort of bizarre interaction. For laughs I tried to run MakeMaker's test suite aggregated and it didn't even make it out of the gate because of an __END__ block in one of the tests. Gravatar::URL worked, but that's a very simple module with very simple tests.
In a browser about the worst that can happen is you've got some left over cookies set, and I think there's a Selenium widget to clear them. Javascript will have no memory between runs by design.
So while the benefits may be the same, speed, reusing a browser process is a natural extension of how a browser works and gets used. Whereas Test::Aggregate is granted no such slack and test authors have to constantly be on guard. This is not meant as an indictment of TA, but an acknowledgment that one is an engineering feat against stacked odds while the other is just using a thing as designed.
Awhile back, I wrote the following tutorial
# Using WWW::Selenium To Test Or Automate An Ajax Website
http://perlmonks.org/?node_id=720018
It is more about getting the most out of Selenium and working around some issues than it is about integrating it into your testing framework. In any event, any tips, tricks or pointers you might be able to share would be very much appreciated.
Re:Any tips, tricks or pointers on WWW::Selenium?
schwern on 2009-06-08T23:47:18
I had none of the problems you had getting Selenium RC running on OS X. It fired right up and found my Firefox. I get no errors switching base domains. I didn't try it with SSL. Maybe they fixed it with 1.0?
I don't have much experience with Selenium, honestly. I've taken over doing acceptance level QA at Open Sourcery and have just started seriously using Selenium. Its all been in the IDE up to this point -- recording and tweaking for Ajax (I've found waitForElementPresent to be very handy in that regard), running them manually -- thus the frantic urge to get this automated and push the testing back down to the developers.
