Rites of Passage

ziggy on 2003-06-20T03:06:34

Every 26.4 days, someone laments in public that there are just too damn many templating modules on CPAN. (The frequency may have increased since the last time I made up^W^Wcalculated that statistic a few years ago.)

This week at YAPC, Damian demonstrated a quick hack he cooked up called makeslides. What does it do? Pretty much the same thing every one else's makeslides program does: convert a text file into a presentation of some sort: HTML, Keynote, PowerPoint, Magicpoint, TeX, PDF or otherwise.

Why is it that there are some areas in Perl where everyone cooks up their own solution, or hacks their own customizations onto someone else's highly customized (but unextensible) solution? It's almost as if you can't call yourself a Perl programmer until you've written either (a) a templating system or (b) a slide generator at some point in your Perl programming career.


The sad truth

Robrt on 2003-06-20T03:21:51

Or more than one of both.

Don't forget an email-address parser.

Woo Hoo!

chaoticset on 2003-06-20T04:33:58

Hey, I accidentally hand-rolled a templating system back during my CGI 231 class! YAY!

Now I just need to write a slideshow-er...

Getting Smarter

chromatic on 2003-06-20T04:48:41

That's why I wrote my own web server.

One More

vek on 2003-06-20T13:12:16

Don't forget the DBI abstraction layer :-)

Re:One More

inkdroid on 2003-06-20T16:32:38

Yeah, I've noticed this one too. I can understand having a Class to create the DBI handle (no need to remember pwords, usernames, etc), but it seems that everywhere I've worked people want to replace DBI's methods with their own (which simply wrap the basic ones). DBI has everything I've ever wanted to do with a database, and then some. Using DBI makes it easier for new people to come in and know exactly what's going on, without having to understand someone else's idea of what database access *should* be.

Re:One More

schwern on 2003-06-26T19:09:36

After you've written your 39823th slight variation on the basic SELECT, INSERT and UPDATE statements, it gets a little tiresome.

Re:One More

barbie on 2003-06-30T17:32:29

There can be good reasons for it.

The wrapper that I continually reuse, implements a keyword lookup into a Class::Phrasebook XML file. This has meant I have been able to move from MySQL to MS SQL Server 7, by only changing the XML file. Plus my wrapper creates the DBI object for you, but only if you require it. I have seen several implementations of CGI scripts that create a connection, even when the request never uses it.

Although Class::DBI may be growing in popularity it doesn't suit all purposes, and sometimes writing a wrapper (hopefully a simple one to understand) is better than recreating all code to access the DBI calls.

If I've misunderstood and you are talking about people rewriting DBI.pm in their own fashion ... then please forgive me. They must be complete mad!

It is the curse...

jordan on 2003-06-20T15:35:43

of TIMTOWTDI!

another rite of passage...

boo_radley on 2003-06-20T17:10:07

"Hey, I could retrieve cartoons from websites with this LWP thingie!" (as discussed on perlmonks a while back)

Re:another rite of passage...

jdavidb on 2003-06-26T18:18:48

Did that. :) And Randall did it a long time ago in one of his columns, but back then it was retrieving Dilbert from usenet, I think.

Its a learning thing...

barbie on 2003-06-30T17:46:29

How often have you writen a "hello world" program? Why, everyone and their dog has written one, why write another? The simple answer is that you needed to start somewhere to learn how to program in a certain language. Sometimes you can learn that by doing the little things, like another makeslides script, you find bigger and better things out there that really do fit your requirements, when at first you didn't think they did.

Others do it simply because their idea of the concept can be quite limited and they want to understand better either the language or the methodogy behind what they are doing.

There are also the people who have a different take on a particular problem and feel they can do it better. Templating systems are a good example here. If Andy hadn't been inspired enough to write another templating system, Template Toolkit would never have seen the light of day. There were plenty around at the time, why write another? Because he had another way of looking at the problem.

If only one app was ever allowed to be written to perform a particular task, I doubt Larry would have been given the go ahead to write yet another programming language.

Through the eyes of database abstraction

schwern on 2003-07-03T06:35:26

So I'd written up a big detailed hypothesis about why database object abstraction layers (Class::DBI, Tangram, Alzabo... stuff on top of DBI) keep getting reimplemented because they share a lot of the same attributes and I know something of the POOP (Perl Object-Oriented Persistence) world, being guilty of reinventing that wheel myself with some success. Unfortunately, in a fit of false laziness, I wrote that on my Win98 machine (its only for games, I swear) and it crashed after writing it.

Reason #19388 why I hate web-based applications: No autosave for textarea boxes.

Anyhow, since I'm not going to write all that again, here's the gist. Essentially, both POOP and templating modules have a very deep, very wide scope that touches almost every part of a system's design. Unlike most modules, which are essentially just a collection of functions and their effect on your program is limited to the area around the point of call, the design philosophies in POOP and templating modules can effect the entire structure of your program. Since they are so pervasive, they can also be a major effect on performance, causing people to avoid an existing module because its too slow. Finally, the problem is superficially simple, anyone can write a very basic tempating or POOP module, but the details are horribly complex to get right. So everyone figures they can knock off a new templating module that does "just what they need" in a few hours.

Here's some specifics to POOP modules.

  • Effects class and object structure. POOP modules essentially take over how classes and objects are implemented. Everybody has their own idiomatic way they want to write objects in Perl, therefore their ideas about how objects are implemented might not mesh with the POOP author's design.
  • Reimplementing a language. POOP modules must eventually do just about everything SQL can do. And do it portably. And smarter. That's a lot to ask for. Most succeed, so somebody tries again.
  • Performance critical. Somebody's always going to want to ram 10,000 queries a second through your POOP system and have it run just as fast as a straight DBI prepare/execute/fetch loop. The more flexible a POOP module, the slower it is (in general) so somebody that needs high performance over features is going to reimplement a thinner, less flexible but faster version. Of course, soon they need more features...
  • Portability. Let's face it, SQL isn't very standard. Its really, really, really hard and time consuming to generate portable SQL. Most POOP modules don't bother and just cover whatever databases they care about (usually MySQL and one or two more). Somebody's going to need to be more portable than that, so they have to write their own.
  • Flexiblity. Because POOP modules have to do so much, somebody's always going to want to do it differently. Or make it do something more. Or make it go faster. So they have to be flexible. But flexibility takes time, thought, effort and often cuts into performance. Often, a POOP module just isn't flexible enough for what someone wants, or adding their new feature is too convoluted or too slow using the existing architecture. Or the POOP module isn't well thought out, being an overgrown "I'll implement what I need in a few hours" module. So, you've got to write your own more flexible version (which might wind up being too slow for others).
  • Doesn't fit your table design. Some POOP modules have strong ideas about how RDBMs should be layed out and only fit certain database designs (guilty). Others want to be able to set up the system from scratch. Still others want to be able to add their own administrative tables and columns. These constraints might not jive with what a user needs, so they write their own.
  • Inadequate docs. Since the scope of a POOP module is so large, they often have inadequate docs for the task. Often they're missing a simple "get me started" guide and the user is overwhelmed with that simple task of just setting up a basic table abstraction. This is primarily why I wrote Class::DBI, the existing solutions just bowled me over.
  • Complex dependencies. Many POOP modules have a very large number of module dependencies (guilty) which would make one blanch away from having their application depend on it. Someone reinventing the wheel would write their own which has fewer dependencies.
  • Complex installation. Similarly, if the POOP module is difficult to install, you dont' want to depend on it. If it asks a lot of questions, requires databases to be set up in a certain way, doesn't install nicely from the CPAN shell, is overly large, etc...

So that's POOP modules. Analogies can be drawn to templating systems, they share the wide scope, the idiomatic designs (especially WRT templating language design) and pervasive design choices. Wide, deep design choices can often turn someone off from an otherwise wonderful templating modules. For example, I hate XML. AxKit is all about XML. So I don't use AxKit. Text::Template requires/allows web designers to write Perl, often unacceptable and even dangerous.

Also, by far the most defining charateristic of a templating module is the templating syntax. And Perl programmers love their syntax! So of course they're going to keep reimplementing their own mini-languages that suit their own needs. I think if someone invented a really solid templating module which allowed language plug-ins, they might have a category killer.

That's how I see it.

Directory Recursion

pne on 2003-07-15T15:45:26

Mine was something that went through directories by calling readdir() recursively. (Never mind that there's a core Perl module that can do this for you.)