Simple math not so simple

Ovid on 2002-11-20T18:08:33

We have a client, who shall remain nameless, who provides data feeds for a few very large companies. However, those companies don't get the datafeeds directly. Instead, a series of small programs that I wrote read in the data (in the form of CSV files) and users who pay a small fee can view the resulting Web pages. The problem: this was a quick hack.

I was originally asked how fast I could get these pages up as our clients were in violation of their contract terms to provide this info, so I took about two days to get this put together. This was done with the caveat, carefully explained to our client, that the data format must not change. I built in a bit of flexibility, but I simply didn't have the time to do this right. The pages have continuously broken since then. As it turns out, there's only been one bug in my code (woohoo!). Every other time things have broken has been due to their changing the data format without telling us. Today was the latest. Originally, the HH:MM:SS in the data file would be in the form /(\d\d):(\d\d):(\d\d)/. Now they've changed that so the above regex fails. It's now /(\d\d?):(\d\d?):(\d\d?)/.

A couple of months ago, they stopped sending timestamps, just date stamps. Sometimes they've changed the data keys without telling us. What I really enjoyed, though, was the time they decided to increase the number of fields they were going to send. Naturally, we didn't find out until the people downstream from us called to yell at us. Our clients keep sending bad data and the people who see it yell at us and we can't sit around and trash our clients by pointing out that they're clueless. We can't win. In fact, this little job may just go away entirely because our client can't seem to send us properly formatted data and the downstream people are getting pissed!

So what's the "simple math" I referred to in the subject? Originally, we had a very, very small bid to put data validation in this project. If anything didn't meet our expectations, we would send email to our client and ourselves and nothing would get updated (and risk breaking the Web pages -- better old data than bad data). We were turned down and now the client has spent 10 to 20 times that cost in tracking down bugs, not to mention the damage to their reputation. Their reason for turning us down? "These are computers. Once we get it dialed in, nothing will change."


Great Lies of Software Development

chromatic on 2002-11-20T20:01:01

"Nothing will change" is in the top five.

Damage to whose reputation?

dws on 2002-11-20T21:27:42

We were turned down and now the client has spent 10 to 20 times that cost in tracking down bugs, not to mention the damage to their reputation.

Sounds like they're paying your company to take that damage. From your client's point of view, that might not be such a bad deal.