Just in time for the dead line to send proposals for YAPC::Israel::2004 I have released version 0.09 of yapcom - the application to manage registrations and proposals.
In this version registrants can already login to the system, submit proposals, edit their own information and their own proposals.
The adminsitrators can see most of the information about every registrant and about every proposal.
I am quite happy that we made it in time but on the other hand I think I am working too hard on this.
It is strange.
In spring, this year when I was working on the registration form for
YAPC::Israel::2003
I used Apache::ASP
and I think I spent a lot less time writing it.
Sure 0.09 of Yapcom already knows more than the earlier web interface but I still have the feeling that the number of hours I invested in yapcom is not proportional to the hours it took for the Apache::ASP version.
Sure, in Apache::ASP I get the session management free but I am not sure if this is the explanation. Back then I used an xml file as the 'database' but actually it was not a good idea as it did not scale at all. At around 60-70 registered people it already started to eat too much memory as it had to read the enire file into memory and lock the file for each change.
Now I am using Class::DBI, DBD::SQLite, CGI::Application, HTML::Template and it seem it is a lot slower to write this code. It is also true that this time I wrote a test suit that already has over 200 tests in it and I feel much safer when I change something.
I think I can also say that this code is much more extendable and while it is not very clean (yet) I do ocassionally refactor it. Sometimes with the help of a number of people.
Last and probably most important: I'd like to thank to Yehuda Berlinger, Shlomi Fish and Richard Clamp for their help with this version.
Well, it seems you have encountered a couple of relatively documented phenomena. The first is the "Law of Diminishing Returns" - the more you work on a project, the harder you have to work to substantially improve it. I.e: you can get a basic, ad-hoc version out very quickly, but if you want to make sure it is real good and add more features, you find it more difficult and difficult.
I don't know what ESR has to say about it in regards to the almost over-the-edge optimism of "The Cathedral and the Bazaar", but it is a well known phenomena in software engineering.
Another thing you've encountered the previous time you implemented the YAPC management system is Joel Spolsky's Back to Basics syndrome (note: the article is nice, but has several major inaccuracies). Basically, by not using a database, you increased the complexity of the operation from O(logN) to O(N). And because of using XML, you also caused a large overhead of parsing and analyzing the data.
Finally: tests. While I liked the tests I wrote for Quad-Pres recently, I found that writing them consumes quite a bit of time. When revamping the IGLU Jobs Tracker, I decided to avoid writing tests, simply because I was "in the zone" and was anxious to add more and more features. I encountered a few bugs, but fixed them quite easily. I'm not sure the time I would have spent writing tests, would have fared below the time spent fixing all bugs and future bugs caused by the lack of them.
Whether or not it applies to Yapcom: I cannot effectively judge. The IGLU Job Tracker is of much smaller scope, and pretty simplistic at what it does.
Heresy
phillup on 2003-11-28T21:52:46
I have tried to be a Good Boy(tm) and write test at the prescribed time... I gave it up.
If I'm web programming, I focus on my code and don't worry about writing tests.
And... I don't write tests... UNTIL I need to refactor, or patch a bug.
Then, I test heavily. Especially if I'm fixing a bug. I make sure I have a test that flags the bug... and I make sure it passes when I'm done.
I haven't seen the gains from testing otherwise with web based code, because it is so much more difficult (for me) to write web based tests... And... many of the "bugs" are CSS or JavaScript or HTML issues and not really "logic" bugs.
If it goes into a module... different story completely.
