All the Perl that's Practical to Extract and Report

First Principles

Robrt on 2002-11-28T04:50:23

For the past few months, on and off, I've been working on a new search interface for one of my favorite tools. The fun part - I wanted it to be a naturalish query language, SQL based. (So, it would be SQL-like => Internal Object Representation => Real SQL).

I've tried many things, from SQL::Statement to Parse::RecDescent. I looked at Parse::Yapp, and even considered shishi. What did those modules get me? A lot of frustration.

This afternoon, after another losing bout with Parse::RecDescent, I threw up my hands.

At that point, I went back to first principles and wrote a simple tokenizer. In goes the SQL, out comes some standardized tokens. The best part? It was easy. Regexp::Common helped with quoted strings, and I was in the home stretch. A simple state machine later, and I had a working SQL-like search tool.

Lesson: Sometimes a pre-existing modules isn't the way to go. It can overcomplicate things and make you work in the wrong direction. KISS works!

and I hope you looked at the other solutions

merlyn on 2002-11-28T16:02:23

like the Text::Query package, which seems to describe itself as solving a similar problem?

Pre-existing modules

ziggy on 2002-11-28T16:10:44

I came across something recently that questioned the wisdom of not reinventing wheels. I can't remember where I read it, but it the tone was something like "reinventing wheels can be a good thing". The justification was that not every problem is identical, and we need problem solvers much more than we need solutions in search of a problem.

I'm pretty sure it wasn't referring to the pathological case of templating modules on CPAN though. :-)

In any case, I'll try and find a citation, but it's going to be difficult. I used to keep track of what I was reading on my Pilot, but haven't used my Pilot in over a year. There's an off chance that it might have been in Pete McBreen's Questioning Extreme Programming or Richard P. Gabriel's Patterns of Software, but I can't say for sure either way. (Both books are wonderful and certainly worth reading.)

Almost...

Robrt on 2002-11-28T18:57:49

I didn't know about Text::Query, but it wouldn't do the job, because I need a tool that can do algebraic combinations of [key op value] triplets. It also suffers, like many many other modules, from anti-documentation-itis. But anyway, my wheel is round enough for now.