Why Port Bugzilla Away from Perl?

chromatic on 2007-05-03T19:38:17

The Problems of Perl: The Future of Bugzilla.

I've heard this song before. The verses are Don't make me install modules!, We're not very good Perl programmers!, and We didn't design the code very well, thus the language sucks!.

You'd almost think we've achieved true AI, the way certain programming languages reach out of our monitors and force us to write unmaintainable code.


I wonder...

sigzero on 2007-05-03T22:10:45

Did he even read his own pro and con list for languages? I did and I came out with "Perl" just by reading the pro/con issues of the other languages.

Irony, anyone ?

renodino on 2007-05-04T00:07:42

Maybe Max should talk to his blog host ? I hear tell they've maintained a few lines of Perl.

bugzilla and perl

rjbs on 2007-05-04T02:27:02

Having hacking on Bugzilla a fair bit, I find it hard to take seriously the idea that Perl is what's wrong with Bugzilla development.

Their codebase is a mess because they lack enforced coding standards. The poster seems to indicate that he thinks this won't be a problem with other languages. Maybe it's significantly easier to maintain a single style of coding in Python or Ruby, but it's sure not just a given. I've also worked on my share of horrible Python or Ruby.

Why not just rewrite parts of the system, as he suggests, but in good Perl rather than some other language that will require bridging.

Re:bugzilla and perl

sigzero on 2007-05-04T11:48:14

If they don't enforce coding standards now; then any language they do choose is not going to help. Python has the "whitespace" thing but if you don't have standards you still get a wide array of coding practices. Ruby gives you nothing more than Perl as far as maintaining style.

Re:bugzilla and perl

Aristotle on 2007-05-04T14:15:43

Yeah, but Perl is line-noisy! And it has regular expressions! And it gives you many ways to do things! It’s even the language motto, “TMTOWTDI”! Whereas Ruby’s motto is “optimise for programmer happiness” or something like that! Asdfghjkl!!1

(Sigh.)

Re:bugzilla and perl

avatraxiom on 2007-05-04T19:58:00

Actually, nowadays we have pretty strict enforced coding standards:



We did lack them in the past, though, it's true. I'd have less trouble if I hadn't been spending the last three years slowly and painfully refactoring all of Bugzilla.

-Max

Re:bugzilla and perl

avatraxiom on 2007-05-04T19:58:46

They're here: Bugzilla Developer's Guide.

Re:bugzilla and perl

chromatic on 2007-05-04T21:01:03

That's a good start, but it focuses on design in the small. That's not a bad thing. Design in the small is very important. Design in the large is also very important, and there's no substitute for good taste (which often comes from bad experiences).

I have the strong, armchair impression that design in the large is your biggest issue right now. Rewriting (or refactoring) is the cure. Porting to another language may seem like it helps, but in my experience getting rid of accumulated technical debt is more important.

Switching away from Perl may be the right choice for other reasons, but the language itself has sufficient abstraction and power that I'm not aware of any design-in-the-large problems it suffers. I could give you a long list for C and PHP and to some extent C++ and Java.

Re:bugzilla and perl

avatraxiom on 2007-05-04T21:23:32

Yeah. We've actually been working on design in the large for quite some time now--I've been largely spearheading the refactoring effort. It's kind of hard to describe those principles in something like a Developer's Guide, so that's why they're not there. (Except in the sense of the General Guidelines.)

It's true that Perl is powerful enough to implement the design-in-the-large features we need. There may be other good reasons to switch to another language or framework, though.

The primary reason I posted the post is to start the process of research. That is, to see really what our best choice is. We're going to do some prototyping in various languages and frameworks, and see if there is a real tangible advantage in any of them.

-Max

Re:bugzilla and perl

vek on 2007-05-04T22:38:24

We're going to do some prototyping in various languages...

I think your time would be better spent reading Conway's Perl Best Practices book and getting familiar with running your code through Perl::Critic rather than attempting prototypes in other languages.

Re:bugzilla and perl

avatraxiom on 2007-05-04T22:54:50

Noted.

-Max

Re:bugzilla and perl

chromatic on 2007-05-05T06:19:14

It's true that Perl is powerful enough to implement the design-in-the-large features we need. There may be other good reasons to switch to another language or framework, though.

Certainly true. If you have expertise with a particular language or framework and your switching costs are less than the cost of refactoring in Perl, it's clearly a good change, and rational people will understand that.

I just hate to see important choices like this made on the basis of language advocacy, and your initial post concerned me. I'm glad to see that my initial impressions were wrong.

Re:bugzilla and perl

avatraxiom on 2007-05-05T06:31:22

Yeah. That's exactly what I'm trying to evaluate, if the switching costs would be less than the cost of refactoring and then future maintenance. :-)

And thank you, I'm glad we worked that out. :-)

-Max

pkg-config

rafael on 2007-05-04T08:31:46

On the other hand, the new BSD pkg-config was rewritten from scratch in an appropriate language, namely Perl (link)...

Re:pkg-config

avatraxiom on 2007-05-04T20:00:00

What was it written in before? C? I'd highly support a move from C to Perl as an improvement. I'd never suggest moving from Perl to C, except in performance-critical areas (and we have XS for that anyhow).

-Max

The difficulty of CPAN dependencies

markjugg on 2007-05-04T15:14:26

The difficulty of deploying Perl projects with CPAN dependencies is a real problem. It's not technically a language problem, but a cultural one.

(PAR even seems like it could help a fair bit with this).

Culturally, we tend to say "Oh, and install this slew of modules from CPAN".

If instead we said "just unpack it and visit install.cgi", then we'd have more users of Perl projects.

I like that a number of PHP projects work this way. It's not a technical benefit of PHP that this trend exists, it's a different culture.

    Mark

Re:The difficulty of CPAN dependencies

chromatic on 2007-05-04T18:37:21

I like that a number of PHP projects work this way.

I haven't installed a PHP application in several years. Does this installation approach run automated tests? (Do these applications have automated tests? Given the customization possibilities of php.ini let alone self-compiled PHP, there are plenty of things to go wrong.)

Re:The difficulty of CPAN dependencies

grantm on 2007-05-04T22:16:55

The difficulty of deploying Perl projects with CPAN dependencies is a real problem. It's not technically a language problem, but a cultural one.

Perhaps, but in my experience it's a solved problem. The team I work with develops and maintains a large Perl code base on Debian. We deploy our code using packages which in turn specify dependencies on packages for CPAN modules. In the event that we need a module that's not already in Debian or we need a later version, we build our own and put it in our repository. The dh-make-perl utility means that for the vast majority of CPAN modules, downloading a CPAN tarball and building a .deb from it boils down to this one liner:

dh-make-perl --build --cpan Module::Name

Of course if we were trying to deploy to 'some random hosting company' then things wouldn't be quite so simple but I don't really see how they'd be any easier if we were using another language. In particular, this has been the major pain point in my recent adventures with Rails.

Re:The difficulty of CPAN dependencies

drhyde on 2007-05-05T19:50:28

Bingo! Give that man a gold star!

Installing perl applications *sucks* even for those of us familiar with perl and how perl people expect us to work with their code. For yer average sysadmin, trying to install a typical perl application is a nightmare. I'm afraid that the perl community's rampant code re-use and insistence both on never re-inventing wheels and also on not bundling other peoples' code hurts us a lot here.

Yer average python application, on the other hand (let's pick the only one I use which I know is written in python - rss2email) doesn't worry about bundling what are, to the pythonistas, standard libraries. And strangely, it Just Works Everywhere.

Re:The difficulty of CPAN dependencies

jmason on 2007-05-08T17:57:16

I think that's just that one app; the other one I'm familiar with, buildbot, has really grotesque upgrability issues in my experience, *worse* than a perl app's. :(

Here, have a look at the log from the last time I upgraded the SA buildbots:
http://wiki.apache.org/spamassassin/BuildBotUpgradeNotes

Re:The difficulty of CPAN dependencies

Lecar_red on 2007-05-08T21:21:02

Not sure but would a bundle distribution help these kind of CPAN dependencies issues?



http://search.cpan.org/user/adamk/Module-Install-0.65/lib/Module/Install/Bundle.pm

In short...

alessio on 2007-05-04T16:38:19

To put it in few word, I'd say he doesn't have a clue. Oh, and by the way Best Practical's Request Tracker (RT) is written in Perl, and extendible with Perl. And in my organisation we moved away from Bugzilla to RT some time ago...

This happened because we needed to run BZ on PostgreSQL and not MySQL, and that alone took years and a few major releases (if it has happened at all) because of the badly-organized *SQL* code. This should say all.

Ruby is the answer?

perrin on 2007-05-04T17:17:50

It's going to be pretty amusing 5 years from now when all the people who were somehow tricked into thinking Ruby would be harder to write bad code in than Perl wake up and smell the pile of nasty Ruby code they've amassed. I wonder what language they'll jump to next.

Haskell! or Erlang!

qu1j0t3 on 2007-05-09T15:45:27

n/t

Rather than the bashing...

speters on 2007-05-04T18:39:32

...wouldn't it make sense to send a few patches. OK, bomb them with patches. The only way their going to see that #11958 applies, is to show them what they've done wrong.

Re:Rather than the bashing...

speters on 2007-05-04T18:42:57

For example, how many problems do you see below in Bugzilla's collectstats.pl.

sub check_data_dir { my $dir = shift; if (! -d $dir) { mkdir $dir, 0755; chmod 0755, $dir; } }

Re:Rather than the bashing...

chromatic on 2007-05-04T19:46:19

Do you count each race condition as a separate error, or do you put them all into a single bucket? The same question applies to permissions.

No examples

jdavidb on 2007-05-08T22:16:51

I like how he lists all the things that are wrong with Perl, but never provides an example of how that "wrong" thing has caused a specific problem with Bugzilla and its development, other than the "lots of modules" problem. Indeed, as you say, languages don't reach out and make people write unmaintainable code. If it's really such a problem, he should've been able to post code clippings illustrating the problems he mentioned. :)