Apparently I have now made the Brainbench test more valuable. The question about the debugger flag is now alternatively a test of whether you're part of the community and read use.perl. (You may debate the detailed merit of this, but see the next paragraph)
The person I was chatting with (who may want to remain anonymous, hence no name yet) suggests that the Brainbench test is of little use, because all it can test is trivia, so it has to test extremely obscure trivia to try to make a differentiation. I now know that the Javascript on the test means that they track the changes you make to your answer, as I'm told that I got only 39 out of 40 questions right, because I changed my answer on the question about compilers from the "right" answer to one of the "wrong" answers. (And in turn I put in the comments after the test that I probably had picked the not-quite as good answer, but that I considered the question flawed as both answers were mostly right, but neither was completely perfect. And they don't even cover things like Solaris::PerlGcc which lets you (mostly) work round the problems of not having the right compiler that your Perl version is built with. And they deliberately rate one answer as right and all 4 others as wrong, even if some answers are partially right.)
I am coming to the opinion that the test is worthless. All that it gives is a way of automatically determining if someone knows their Perl trivia. It doesn't test actual understanding at all. It doesn't test how someone thinks. And I agree with alternative that the person I was chatting with suggests - a well thought through set of e-mail-able screening questions can elicit answers that you can skim within a minute to decide whether to bin a CV, or procede further with that candidate. Given the volume of CVs that Perl employers get, I don't think that a 1 minute cost per candidate is a problem, particularly if it lets you bin a couple more candidates that Brainbench would show up as "maybes", such that you had to spend the time actually interviewing them.
Years ago I took one of their tests and they sent me a certificate listing me as a 'certified Perl programmer'. However, I never listed that on my CV since I care about whether someone can get the job done and look up things quickly, not how good their memory is. My memory is awful and if I don't take copious notes about something or do that something over and over again, I simply forget it. I can't imagine how a Brainbench test would be valuable in the face of that, but someone with worse skills but a better memory is going to fare far better on that test than I.
Here's an idea: how about a test whereby a test script is sent to someone (along with a possible list of coding standards) and they need to make it pass? The judging criteria would be how long it takes them to return the code and a possibly subjective evaluation (or Perl::Critic?) of the code quality. Admittedly, this isn't perfect, but it might be better than Brainbench.
In other news, I just read about your potential $employer asking you to take that test. I laughed myself silly. Thanks
Re:BrainBench?
nicholas on 2006-09-15T11:19:28
I'd much prefer certification from Klortho the Magnificient, but I believe that he's no longer in that business.
:-(
