All the Perl that's Practical to Extract and Report

What does it mean to "know" Perl?

autarch on 2007-05-07T02:01:40

One of the Bugzilla maintainers posted a blog entry on "The Problems of Perl" (on LiveJournal, love the irony ;)

It's spawned a lot of discussion and some advocacy (ugh, advocacy). One of the points I've seen come up several times is that the author complains about something in Perl, and someone suggests a module that might help, to which the author replies "I'll check that out."

In a few other threads the author has claimed to know Perl pretty well.

In reponse to that a few folks have said "well, if you don't know about Module X you don't know Perl."

This brings up the question of what it means to "know" Perl. Personally, I tend to agree, this blog author really does not know Perl. He may be able to read Perl well, and he may be able to write clean Perl, but that's different from really "knowing" the language.

A programming language is more than the sum of its syntax. In that sense, I know C. But in the sense of actually understanding the lore and spirit of the language, I don't know C at all. But the word "know" is pretty loaded here, and it's easy to interpret in many different ways. Let's try to come up with two different terms.

For someone like this blog author, I'd say that they are "competent" with Perl. They won't be flummoxed trying to read Perl, if you give them a task to code in Perl they can do it without writing something horrible, but they won't do the best possible job either, because they don't know enough to do so.

For the other version of "know", let's say "expert". An expert knows more than the syntax, they know the lore. They're aware of the ongoing community and discussion behind the language. For Perl, a large part of this means being aware of what's on CPAN, and knowing how to use it to solve problems. It probably also means going to mongers meetings, conferences, reading mailing lists, hanging out on Perlmonks or IRC, etc. You don't have to do all of those, and you don't really have to be an active writer/speaker, but you have to have some level of involvement.

Achieving competency should be a pretty quick process for anyone who's competent in another language. Once you've learned the syntax and some core libraries, you can apply your existing design skills to write not-too-horrible code.

But achieving expertise is much harder, and it's an ongoing effort. Right now I'm definitely a Perl expert. If I went away for two years and came back, I'd probably just be competent, and it'd take catching up to become an expert again.

In practical terms, I'd prefer to work with experts than competents, but I'd hire a competent if I had enough experts on the team already to make sure the competents do expert-level work. A team of competents will produce an okay code base, but one that reinvents many wheels, and has a lot of niggling problems.

Unfortunately, it seems like employers can't even distinguish between competent and incompetent, much less between competent and expert. But that's a whole 'nother journal entry.

Re: What does it mean to "know" Perl?

vek on 2007-05-07T06:11:27

Right now I'm definitely a Perl expert.

And modest too.

Re: What does it mean to "know" Perl?

sigzero on 2007-05-07T12:09:02

I don't think he meant it in an arrogant way. I think that was a summation defined by the criteria given in the previous paragraph. Given those criteria the author considers himself "expert" level in Perl.

I myself, would be competent, striving for expert.

: )

Re: What does it mean to "know" Perl?

petdance on 2007-05-07T15:50:38

There's nothing immodest about calling oneself a Perl expert, especially if you're Dave Rolsky.

Re: What does it mean to "know" Perl?

vek on 2007-05-07T17:24:11

I'm well aware of who Dave Rolsky is. Thanks.

Re: What does it mean to "know" Perl?

autarch on 2007-05-07T16:14:57

I have never claimed to be modest, nor would I. Given my giant ego, saying that I'm merely an "expert" in Perl is modest compared to what I might want to say ;)

perl purity?

slanning on 2007-05-07T14:26:47

Someone (Tom Christiansen?) wrote a description of different levels of Perl experience, from novice to user, hacker, guru... (I see a "Perl Purity" test, which uses the levels, but I thought there was a description of each of the levels. Maybe I'm thinking of something else.)

Re:perl purity?

grinder on 2007-05-07T15:03:53

Nat Torkington is the person you were thinking of: Seven Stages of a Perl Programmer.

Knowing Perl

petdance on 2007-05-07T15:52:36

The whole idea of perl101.org, which we're still trying to get in a form I won't mind publicly announcing, is "What Every Perl Programmer Should Know." It's the bare minimum bar of what to know to take full advantage of the language in the cases that I think people will be using it.

So by that definition, the answer to your question is "If you know everything in perl101.org." :-)

CPAN Expertise

chromatic on 2007-05-07T18:58:17

For Perl, a large part of this means being aware of what's on CPAN, and knowing how to use it to solve problems.

That was much easier five years ago. By that criteria, I have a difficult time considering myself a Perl expert; it's awfully difficult to stay up to date on the best CPAN modules when there are so many new ones every week.

Re:CPAN Expertise

jhi on 2007-05-08T01:46:09

If "staying up to date on CPAN" means "being knowledgeable of the latest templating system / CMS / framework / OO gimmick in CPAN" I lost count, umm, many, many years ago. Because I don't use any of them.

Re:CPAN Expertise

chromatic on 2007-05-08T04:19:34

It could mean that, or a DBIx plugin, or which Unicode-aware module handles encodings correctly and transparently.

Re:CPAN Expertise

autarch on 2007-05-08T05:32:45

I meant it more like "knowing what to use". As far as template systems go, I don't think the best modules in that class have changed for a long time. There's TT, Mason, and HTML::Template. For some (non-web) uses, there's also Text::Template. Anything else I would ignore, except maybe Jesse's Template::Declare, but that's far from mature enough to be best in class, and has yet to prove itself.

OTOH, if you're writing your own templating system or using eperl, you're not up to date on CPAN ;)

Re:CPAN Expertise

autarch on 2007-05-08T05:30:11

Yes, there's many new CPAN modules all the time. I "follow" the new uploads, as in I look at what's new at least once a day, and occasionally click through to read a module's docs.

That's a little different from staying up to date on what's best. I don't think there's a new best module in any category every day, by any means. The best of any given category changes relatively infrequently, and it takes time before a new module becomes the best.

By definition, for a module to be "best in category" it has to have been around a while, widely used, etc. Of course, that's relative to the user base for a category. No doubt there are many modules that are best in category because their category has only a few users, and there's only one module for that task.

Re:CPAN Expertise

nik on 2007-05-08T10:18:39

I've found that subscribing to the recent uploads RSS feed helps with that. It makes it pretty easy to keep up to date with new releases of modules that you might already be using, and can be an entertaining 5 or 10 minute diversion when someone uploads something new and useful. That's just enough exposure so that, when 3 months down the line someone says "I need a program to do X" I remember that I've seen Foo::X go past on CPAN.

Max's Response

avatraxiom on 2007-05-08T00:38:14

I actually have heard of and did know about most of the modules and frameworks that were suggested by many of the commenters on my blog, as do several other Bugzilla contributors. Remember, Bugzilla is written by a community, not by just me.

When I first started to work on Bugzilla's code several years ago, it was not in a state where a project like "Use DBIx::Class and Catalyst" was a feasible project. A huge amount of cleanup was required before anybody could even see the architecture of Bugzilla.

I'm glad to take a look at some of these modules now and consider them.

One of the differences between Bugzilla and many other Perl projects though, is that Bugzilla is actually something that a lot of people want to install on their own machine. There aren't a *lot* of people who want to install LJ or Slashcode on their own machine, and if they do they're usually experienced sysadmins with a background in Perl.

So if we start to add lots of required modules to the Bugzilla project, that's a big problem for our users. I know that to you and every other Perl guru I've talked to, installing modules is not a big deal. It's not a big deal to me, either. But to many users who want to install Bugzilla, it *is* a big deal.

I also know we can bundle the modules, and we actually are going to probably start doing that.

Think of the post as, "If I were writing Bugzilla today from scratch, would I write it in Perl?" I certainly wouldn't. So is there some advantage to looking at the *possibility* of moving to some other language? Sure, research never hurt. It's also sparking a lot of interesting suggestions from the community, which I really appreciate.

Also, many of the people who attack Bugzilla's code base haven't seen it in many years, and should take a look at what the current CVS HEAD looks like. I agree it's not ideal, but it's a HUGE improvement over Bugzilla 2.16.

-Max

Re:Max's Response

autarch on 2007-05-08T05:35:30

The module installation problem is real. Yes, Perl folks in the know are comfortable with CPAN, but it's true that many aren't.

As have been mentioned in various threads, there are solutions to that, including "bundle them manually", PAR, automating the install off CPAN, distro packages, etc.

This is not a language problem. Ruby and Python will both have similar issues. PHP doesn't because nobody uses any modules, they just write everything from scratch.

Reiventing all the wheels is of course another viable option, see PHP for innumerable examples.

Re:Max's Response

avatraxiom on 2007-05-08T05:50:49

Hahahaha. :-) Yeah, I hate being stuck between that divide of "reinvent all the gears, wheels, and rods" or "require more modules than anybody wants to deal with." :-)

It's true that it's a problem in any language, though some languages have things like email handling bundled in their standard library.

That's also one reason why PHP users require few modules--they have a ridiculously extensive standard library.

I agree that it's better to spread the work out on libraries across the community, though. It just means that people have to install modules.

-Max

Re:Max's Response

Aristotle on 2007-05-08T19:19:25

they have a ridiculously extensive standard library.

Except when they don’t (because ¾ of it are optional at compile time).

Re:Max's Response

avatraxiom on 2007-05-08T19:38:09

Yeah, good point! :-) I've run into that from time to time. Thankfully I'm usually using the packaged versions that either have them all built-in or compiled as separate packages that are pretty easy to get with the packaging system of whatever distro I'm using.

They also have PEAR, which works pretty well.

-Max

Re:Max's Response

TeeJay on 2007-05-08T14:23:54

I'm not sure how it can be easier to port to a new language that the developers and users know less, than to a framework written in a language the system is written in and developers and users already have some familiarity with.

As it happens, if I was to write Bugzilla again tomorrow, it's not the language I would change, it's the database schema, relationships and workflow - I'd be looking at closer integration with project documentation (specifications, screenshots, UML documentation), version control and release management.

Re:Max's Response

avatraxiom on 2007-05-08T19:35:49

Hi TeeJay. I think more of our users know Python than Perl, but it's true that more of our developers know Perl than Python.

Have you looked at the database schema of Bugzilla in any depth?

We're working on custom workflow right now.

Our goal is to be a bug-tracker, not a project management system, so we don't work on those other things. If we had three times as many programmers, and we had them consistently available, we could consider branching out into those project-management areas, but right now our focus is on bug tracking.

-Max

Re:Max's Response

vek on 2007-05-09T04:06:46

So if we start to add lots of required modules to the Bugzilla project, that's a big problem for our users.

Six Apart have solved this problem rather nicely by bundling all modules with MovableType. It's a breeze to install too.

Off the top of my head, you could also take a peek at how Krang, Bricolage, and *cough* RT *cough* have handled the module install issue.

Re:Max's Response

avatraxiom on 2007-05-09T15:56:35

Thanks! :-) I'll check all those out.

Should A Language Require So Much Experience?

avatraxiom on 2007-05-08T00:49:42

Also, I wanted to respond to your point about how much time and research it takes to become a Perl expert. I agree--that's what it takes.

It probably takes that for C, as well, although I wouldn't know since my C experience is somewhat limited (although I can read C and could pull something off in it if required, and have done a few small projects in it).

However, that's actually one of my primary objections to Perl--that it takes so much. You could say, "Every language takes that long!" But I don't think it does. C may, Java may (less so), and Perl certainly does. But working toward Python best practices takes far less effort, I'd say, than equivalent Perl best practices, partly because the Python standard libraries are more extensive.

I haven't used Ruby enough to know how it is in that community, so I can't say.

Most of the people who are disagreeing with me have used Perl extensively, and haven't used any other programming language as much as they've used Perl.

What I'd be most interested in hearing is, "Yes, I have used both Perl and Language X *extensively* for *large projects*, and here are the advantages I find in each." Nobody's said that yet. I think it may be rare to have such depth of experience in more than one language.

-Max

Re:Should A Language Require So Much Experience?

chromatic on 2007-05-08T03:40:46

But working toward Python best practices takes far less effort, I'd say, than equivalent Perl best practices, partly because the Python standard libraries are more extensive.

I'm not entirely sure that that's true, nor even that that's the most important reason, but I do think you've reached a truth.

Some of the defaults in Perl 5 are wrong. Some of them (particularly in the area of OO) are wrong in the sense that they don't work (SUPER:: is particularly broken for all but the simplest uses). Others are wrong merely in the way that they provide the barest minimal default.

It's good that a knowledgeable Perl programmer can mold the language to a local maxima with a minimal amount of work, but it's not good that said programmer should always have to.

There should be more than one way to do useful things, but it would be nice if the default way were nicer and easier and, most of the time, the right thing without tweaking.

(Yes, I'm a Sixer; how did you know?)

Re:Should A Language Require So Much Experience?

avatraxiom on 2007-05-08T05:42:20

Yes! Thank you, I think you said that very well, and I completely agree. :-)

-Max

Re:Should A Language Require So Much Experience?

TeeJay on 2007-05-08T14:39:29

There are some quirks and issues when you get deep into perl as you've said, but bugzilla doesn't come anywhere close to this, and I'd wager you'd find equally problematic issues when you reach a similar depth of programming in Python or Ruby.

I've had a quick look, and bugzilla still lacks any real OO design, wouldn't know a design pattern if it jumped up and down shouting "I'm an iterator" and still suffers badly from NIH.. heck even copy and pasting CPAN modules directly would be better than some of wheel inventing I saw when I did a cvs checkout to look at adding pagination (yes, pagination.. after nearly 10 years, you still get all the results in a single page).