Blogs. The correct place to report bugs.

nicholas on 2008-08-26T08:36:38

"Your random blog" has never been the right place to report a bug*. So to keep with the spirit, here's a fix to a bug, reported on my random blog.

It seems that there is still a problem with RedHat's packaged perl 5.8."8"**. RedHat seem to have an aggressive policy of incorporating pre-release changes in their released production code. This would not be so bad if they actually communicated back with upstream (i.e. me and the other people on the perl5-porters mailing list), or demonstrated that they had sufficient in-house knowledge that they didn't need to. But evidence suggests that neither is true, certainly for 5.8.x

Let me stress that there has never been this problem in any released Perl, 5.8.7, 5.8.8, 5.10.0, and it won't be in 5.8.9 either when it comes out. The problem was caused by changes I made in the 5.8.x tree that RedHat integrated. End users reported the first bug something like 2 years ago, and RedHat closed it as "upstream patch" rather than reporting back "you know that pre-release change you made, that we integrated - well, it seems to have some problems". So I fixed the cases I was aware of.

I'm human, and it turns out that that it wasn't all the cases. Once I was made aware of this (by a RedHat user, note, never the RedHat maintainer) I fixed the problems he reported, and went on a 2 day trawl of CPAN‡ to locate every other idiom that was going to be a problem.

For their versions affected, RedHat merely need to put out a patch integrating changes 31996, 32018, 32019 and 32025 which FIX IT, are documented as FIXING IT, and are from NOVEMBER 2007.

Works on my machine. And anyone else's machine who is running my code. Particularly my released code.

Although, the better fix is not to use the vendor's Perl release for your production systems. Render unto Caesar the things which are Caesar's and for yourselves, compile your own, and your own module tree, from source you keep and control in your own version control system, which changes only when you change it. In particular, if you're not using ithreads anywhere you should compile without ithreads support, which most vendors choose to enable. (It's not the default, and it costs about a 10% slowdown). Anyone doing this would never even have known that there was a problem with some vendor's interpretation of perl.

Update

No, it's still broken in RHEL5. root had changed everyone's $PATH ("it is my machine, after all"), and in my haste I'd not realised that I actually just typed perl, not /usr/bin/perl. So to be fair they are still asleep on the job. Where's I'm awake and doing this stuff for free.

* Nor is your bug tracker. Upstream's own bug tracker is the O(1) place where upstream reads about bugs in upstream's own software. Not O(n) other people's bug trackers. The latter does not scale.

** At least in some of their distributions. The only RedHat box I have access to, and that's an account created 3 minutes ago, is "Red Hat Enterprise Linux Server release 5" and their supplied /usr/bin/perl doesn't have the bug.

† It has been a different matter for 5.10.0 in Fedora. For that, the maintainer has been very communicative, and so we were able to help him fix problems and get Perl 5.10.0 into Fedora Core 9.

‡ This was when I had a lot more free time than now, mainly because I was having a break between jobs.


To be fair..

TeeJay on 2008-08-26T09:05:25

The blog about this bug wasn't reporting it, it was highlighting the fact that RHEL *still* hadn't fixed it, and gave some interesting background, including links to the thread on the dbic list about it.

We happen to be affected by this bug and it's really annoying - a 10% performance loss from ithreads isn't really an issue as our servers are almost entirely I/O or DB bound, adding DBIx::Class makes for sucky performance on the dev server, but fortunately our production servers don't have the problem.

It's tempting to install our own perl, but compiling perl and recompiling and installing all the CPAN packages on every live server is no small job and one thing we can do without.

Re:To be fair..

jplindstrom on 2008-08-26T11:08:33

Do you really need to compile the modules on every box?

For the project at work, we install almost all CPAN deps in an app specific directory under a local user account. Those files are version controlled together with the application source.

Release basically consists of tarring up the source tree (including deps), and release is untarring things in the correct place and running a configuration script.

If you have different architectures on your boxes this obviously won't work without some extra maneuvers.

We do this for > 95% of the modules. Some modules are very difficult to install without being root, so they are dealth with separately on a host level.

This approach leads to each deployment of all versions of all apps being almost entirely decoupled from each other. Otherwise any upgrade of any centrally installed CPAN module would reqiure a complete retest of, like, everything. Which wouldn't happen. Which would break things eventually.

Re:To be fair..

drhyde on 2008-08-27T10:42:15

Yeah, it's a pain once you've started doing things The Wrong Way to switch to doing things The Right Way. However, it's a once-off pain, and isn't as painful as continuing to do things The Wrong Way.

Yea Verily Amen Forever and Ever

pudge on 2008-08-26T16:18:09

I always compile my own perl for dev and production, and threads is the #1 reason why. I have a new #2 reason, due to RH. Thanks, RH! :-)

We use RH and never knew about the slowdown on Slashdot (or use Perl ... if we had been using the system perl, when we moved use Perl to this new box last week it would have been terribly, terribly slow).

2 Day Trawl

bill_mcgonigle on 2008-08-31T23:44:26

Just a shout out to let you know we appreciate your hard work out here!

10%?

hachi on 2008-10-09T00:39:00

It's not that I don't believe you... I'm actually curious what the 10% overhead is for ithreads.

Do you know of a doc of this already?