Just finished interviewing two programmers. One we thought of as a junior developer and the other seemed to be much more mature in his coding ability. As it turns out, the "junior" turned in a much more mature program for his coding test and, though he had a slight bug, I was pleasantly suprised. The one I thought would be better just threw a bunch of code in a while loop, didn't modularize anything, used 'chop' instead of 'chomp', didn't know how to use Perl's time functions (though he had both Perldoc and Programming Perl available) and generally turned in something that I simply wasn't too impressed with.
The test, incidentally, was basically to display a menu which allowed you to list a log file (in the format of your choice), add a datestamped entry to the log file or exit. Pretty simple stuff that I whipped out in five minutes. Both of them took almost half an hour :(
Also, did you make them write it down on paper? I absolutely hate trying to program in pen. I've been tempted to walk out of interviews when I see them take out the xerox of questions...
There, I feel much better now. Maybe I should start using my own blog.
-sam
Re:Test taking ability
lachoy on 2002-09-04T10:08:05
Actually, I think you probably would be better for the job. I'm a good test taker as well, but this is limited to the classic multiple choice tests where the elimination strategy works 80% of the time. The sort of test discussed here is a little different: if you can sit down with a manual of a language never seen and whip out a working and correct program then you can probably pick up everything you need about the language to get your job done. Yeah, you're missing idioms and other features you can really only get through experience, but it's a good bet you can pick those up quickly too.
I agree 100% about pen/paper
:-) Re:Test taking ability
samtregar on 2002-09-04T15:02:16
Well if that's the kind of programmer you want then you might as well test for it explicitely. You could create a manual for a fake computer language, give the applicant an hour or so to read it and then have them do a few test programs. This was actually done in my CS 202 class at college and it was the most fun I've ever had in a test.But I'm still really concerned that you'd be missing out on some fantastic programmers that have a different psychology.
-sam
Re:Test taking ability
Ovid on 2002-09-04T15:44:45
I can almost see your point, but the nature of the questions that we pose to candidates are so basic (e.g.: what's the potential performance impact of nested loops?) that I am very concerned about someone's inability to respond to them. In contrast, in the Perlmonks thread I started on this topic, someone mentioned writing a routine to go through a binary tree without recursion. While this was described as simple, I find that I encounter nested loops far more frequently than binary trees. I could easily write the binary tree program -- with a copy of Mastering Algorithms by my side. I find I usually don't need them in Perl. When I recently needed a graphing solution with adjacency lists, I pulled out Mastering Algorithms again. I want to memorize the simple stuff and look up the complicated stuff.
As another example, if we were hiring a DBA, I'd ask them to describe the first three rules of normalization, but I'm going to be less concerned if they can't describe the fourth and fifth normal forms well as these are rarely seen in production (plus, I've seen database books that argue about what these forms really are).
Admittedly, there may be some great programmers out there that we miss, but I tend to follow the Pareto rule which states that 80 percent of your results stem from 20 percent of your actions. I might miss a great programmer here or there, but so far, those programmers who I respect have had no problems with those questions and those who I've seen struggle seem to less skilled.
Of course, to totally destroy my argument, one of the programmers who totally fumbled the nested loops question actually wrote a rather nice test program
:) It had some issues, but given his time constraints, I think we definitely would have hired him were we to have a position for him. Re:Test taking ability
Ovid on 2002-09-04T14:41:07
Good question. The questions are here and we do ask that they sketch out the answer to the second question. The rest we can just discuss as I want to know how they think, not necessarily whether or not they get the answer right. Now, if I had difficult questions, I think this could be more problematic.
As for test taking ability, to a certain extent, some of us could argue that this is what all programming is about. The boss runs in screaming "Oh shoot! The knerflebitz aren't fribitgizting! What do we do?" That, to me, is a test. There's time pressure, you can fail, and you wonder why you're sitting in front of the teacher instead of out playing. If you can perform well under pressure, I like that.
