Yesterday, I noticed a behavioral regression on a screen I'm working on, but no tests were failing. Today, my first task was to track this down. I found the problem, wrote the fix, and then realized my tests were still passing. So I added tests to make sure I was covering the problem. Then my tests passed and I had an interesting epiphany. Since I wrote my tests after I wrote the code to fix the bug, I had no way to ensure that I was really testing the bug fix.
I backed out my changes and and my new tests were indeed failing -- but they weren't my expected failures. Instead, I had written buggy tests! I fixed the tests, ensured thay failed in ways I was expecting, put my changes back in, made sure the tests passed and checked in my work.
We're now at 3,628 tests on the most complicated application I've ever worked on. We have no code ownership. I might do a quick fix on something that I know better than someone else, but everyone is pretty much free to work on what they will. Somehow, it works, the app is pretty good and development is still rapid.
Testing, without a doubt, is the single best thing I have ever learned about programming. Give me a programmer who only know baby Perl and if I can teach them testing, I'll gladly work with them over the ubercoder who won't test. I should know. The baby Perl programmer sits about twenty feet away from me.
Q: How do you know your tests aren't buggy? Do you write tests for your tests? Har har!
A: No, I write my tests first. I make sure they fail. Then I write code to make them pass. The tests test the code and the code tests the tests.
Re:Test First
jplindstrom on 2003-05-22T18:49:29
Do you _always_ do this?
Don't you sometimes write something that is so complicated that you can't really predict the outcome (which is why you wrote code to compute it in the first place), and just use whatever you get as the baseline output for future regression tests?
Don't you sometimes find that the API spec of a method evolves in a good way as you go from caller (tests) to method API design to implementation of the method?
I know I cheat. Sometimes a lot.
Re:Test First
chromatic on 2003-05-22T19:49:29
When I cheat, I get caught and I feel stupid, so I tend to cheat a lot less as time goes on.
I can't remember the last time I worked on something so complicated that I couldn't attack it in small chunks. That's another nice benefit of test-first: you have to work in baby steps. My most recent project has methods of three to ten lines apiece. The longest method is 19 lines, and it has the only if-else block in the whole program.
Granted, it's not a complicated problem, but it's a surprisingly small amount of code composed of surprisingly small pieces. This happens just about every time I do test-first development.
Re:Test First
djberg96 on 2003-05-22T21:45:22
Indeed, I consider this to be the best side effect of test-first. It leads to better code design, because you break things down into easily testable methods.I can't remember the last time I worked on something so complicated that I couldn't attack it in small chunks. That's another nice benefit of test-first: you have to work in baby steps. My most recent project has methods of three to ten lines apiece. The longest method is 19 lines, and it has the only if-else block in the whole program.
