Why I don't use the perl Sun ships

jdavidb on 2006-12-08T16:02:10

For reference purposes, for the next time I get asked this question.

(Note that "Perl" is a language. "perl" is an executable, a program, which is an interpreter for the Perl programming language. PERL is nothing. There is no such thing.)

For starters, noone uses just perl. The entire point of using perl is to gain access to the CPAN, a gigantic repository of free code modules and libraries to handle just about every protocol and API in existence. All of our code depends on at least one CPAN module; most depend on many (10 or more). Sun's perl certainly doesn't ship with these modules, and so it becomes necessary to install them.

But once you install these modules, you have touched Sun's perl installation. Yes, there are separate directories within the installation for your locally-installed modules. But you've still touched things, and you don't know what differences that may cause. I consider Sun's perl (or the perl shipped by any other UNIX vendor) to be a part of the OS. I don't know what scripts in the OS itself may depend on that perl, and on its modules. When that vendor ships me additional software to install, they may be relying on features of their regular pristine perl installation which I might have modified. The vendor may provide some support for installing modules with their perl, but it is usually not a high priority, the vendor may not know as much about the subject as I do and hence may be unable to help me if I am stuck, and in general the cost of such support (even if it is free) is too high.

Furthermore, matters become worse when you need to use a later version of a module than the one the vendor shipped. perl currently has no (decent) method for handling multiple installed versions of a module. You can install such modules in a separate location, but if you are going to go to the trouble to maintain that, you might as well be maintaining a separate installation of perl, which is what I advocate.

In short, using the vendor-supplied version of perl that comes with the OS becomes a potential maintenance nightmare.

Also, what happens when you want or need to use a later version of perl than the one your vendor supplies? Sun has been very good for awhile now about providing the latest version of perl, but this was not always the case (and it is not always the case with other UNIX vendors). What if you need features of a perl interpreter, such as threading, that were not compiled in by the vendor? What if you are using code that is incompatible with features, such as threading, that were compiled by the vendor? You really, really need your own perl.

What happens when you upgrade your operating system and the vendor-supplied perl gets upgraded as well, and suddenly all of your programs are incompatible? You may find them emitting warnings they didn't emit before. You might find code constructs that don't work anymore, although this is rare. You might find all of the modules you installed with the vendor perl have disappeared, or been upgraded, or been downgraded.

Wouldn't it be better to have each program tied to a specific version of perl, in fact a specific installation of it with specifically installed modules, and only move it to a newer version when you have the time to test it and verify that everything works? Under the practices I recommend for perl installation, you may install as many independent perl instances as you desire. Each of them may be a different version, with an independent set of installed modules. Your programs are hardwired to the exact instance of perl that you used when you developed them, and if you would like to upgrade you may point your program at the newer version and try it out, then make the change permanent. Once all of your programs are migrated, you may remove older, unused perl installations. This just isn't possible with the vendor-supplied perl. You need your own perl. In fact, you may need several of them.

Also, under the practices I recommend, your instance of perl is in a completely self-contained directory. If you would like to put an identical installation of perl on multiple hosts, you may tar up the whole thing and just copy it (provided everything is the same on both hosts). In many cases this is not the best thing to do, but there are some cases where some people have found it desirable. (Personally, I try to install perl separately on each host, just in case there are minor differences from machine to machine. But if I were using a big cluster or grid or something, I'd probably want to automate the production of ONE perl master installation, and replicate it from host to host, possibly using the operating system's packaging facility rather than just tar. And sometimes people who I don't support call up and say, "Hey, can you give me perl?" and I want to get it to them quick-and-dirty.)

These are the main philosophical reasons I believe in maintaining our own instance(s) of perl. But here's a very big practical reason, specific to Solaris: 99% of the machines I've seen use gcc as their primary C compiler, and do not have Sun's C compiler installed (although Sun provides a helpful stub named cc that tells you the compiler isn't installed). However, 100% of Sun's provided perl installations are compiled with Sun's C compiler, and this means that the module installation system is configured to use this compiler, too. Yes, some perl modules have their guts written in C for efficiency, or need to interface with an existing C (or other language) API. (Think of the interface between perl and Oracle.) So compiling many of the most important modules requires a functional C compiler and a configuration of perl that knows where to find and how to use that compiler. Sun's perl is effectively broken on systems that do not include Sun's C compiler. (And even in the rare case that you have their compiler, using gcc is probably a better bet, because it is more actively developed, more standards compliant, and most importantly more likely to have been used by the developer of the module you want to install.)

Now, it is possible to hack Sun's installed perl and make it see and use gcc and ignore the broken Sun cc stub. I've done it many times. But curiously, I've never done it for myself. I'm always doing it for someone else who calls me up, someone who wants to ignore my strong recommendations about installing their own perl. And every time, I have to remember how to do this again, make sure nothing's changed, do some googling, do some tests. It's a fairly easy process, but it's ugly. And once you've done it, you've changed your pristine vendor supplied perl -- and created a potential maintenance nightmare. What happens when you upgrade, and all those other questions I asked above? Sun may be providing some limited support for doing this now. But can you, or they, guarantee that your code is always going to glue together flawlessly when it was built with multiple compilers like this? It's always worked, so far, but will it continue?

In fact, I'm seriously thinking I may not help such folks anymore. Maybe they need to do the Googling to learn how to do this themselves. (Maybe I just need to make a record here next time I do it, so I can send them the link. Oh, wait; I'm thinking about not doing it anymore.) Or maybe they need to listen to what I'm saying and make their own perl. It's not that hard, it's educational if you've never done it before, and it eases so many maintenance issues. In fact, it is really for these people that I've written this little essay.

In summary, did you notice how many times I used the phrase "maintenance nightmare" above? Why don't I use the vendor-supplied perl, if I can help it? Because doing so leads to many real or potential maintenance nightmares. There's a slim bit of maintenance overhead in rolling your own perl, but this cost is far worth the benefit of maintaining your own, according to your own needs.


perlgcc

thinc on 2006-12-08T18:55:39

Mabye you could try pointing them to perlgcc?

http://use.perl.org/user/thinc/journal/30000

Re:perlgcc

jdavidb on 2006-12-08T20:43:35

Thanks! I knew the guy from Sun (Alan Burlison) had done something to make this easier, but I didn't know what.

My point exactly

clscott on 2006-12-08T19:14:37

I recently wrote a post at perlmonks advocating why you should always install your own perl and how you may run into issues with a vendor perl with a complex perl application.

You've been able to shed a bit more light on why it's necessary, thanks for taking the time to write this out so eloquently.

Re:My point exactly

jdavidb on 2006-12-08T20:40:19

Thank you. I'm glad to hear that my ideas were helpful, and even more glad just to hear someone say, "I agree," so I know I'm not crazy. :) Also, I appreciate your comment in your perlmonks post that your deployment should be testable, repeatable, and automatable. That's a large part of what I'm going for here.

I'm about to post a short followup journal entry to this one that may interest you, too.

*always* install your own perl

jhi on 2006-12-08T20:40:57

That is the only way to guarantee stability, both for your application(s) and for the vendor's application(s) and operating system.

Re:*always* install your own perl

jdavidb on 2006-12-08T21:14:26

Thanks, Jarkko. Having the endorsement here of a former pumpking is probably the best help I could ask for in making this case to coworkers in the future.

Hmmm

Abigail on 2006-12-15T08:29:13

It's been a while since I last used Solaris, but I never have encountered the problems you describe. Unlike most of the Linux distros I've worked with, Sun comes a long way of insuring that its own Perl dependent utilities still work if you change the default installation of perl. The main idea being that /usr/bin/perl is a link to someplace else. All the OSses scripts use the path of where /usr/bin/perl links to, leaving the admin free to replace /usr/bin/perl. It won't break any of the OSses scripts.

A for the Sun pro compiler vs. gcc. The only code that I have encountered that compiles on gcc and not on Sun pro is code that uses gcc specific 'enhancements'. In my experience, Sun pro compiled code is at least as fast as code compiled with gcc, and often faster. Some years ago I benchmarked perl (compiled with Sun pro vs compiled with gcc). The Sun pro compiled perl was 10 to 20 procent faster.

As for gcc being more actively maintained, that may be, but does that help people running Sun hardware? Sun hardware had gone 64 bits (even for their low level stuff) for quite some time before gcc was able to generate 64 bit binaries on a Sun platform. The Sun compiler had no problems with it of course.

Given the choice, I'd pick Sun pro over gcc.

And in general, I would pick a native compiler over a general compiler. I remember Mereijn doing tests on AIX, comparing gcc vs the IBMs native compiler, and finding the perl compiled with gcc significantly slower.

Re:Hmmm

jdavidb on 2006-12-19T22:39:14

It's been a while since I last used Solaris, but I never have encountered the problems you describe. Unlike most of the Linux distros I've worked with, Sun comes a long way of insuring that its own Perl dependent utilities still work if you change the default installation of perl. The main idea being that /usr/bin/perl is a link to someplace else. All the OSses scripts use the path of where /usr/bin/perl links to, leaving the admin free to replace /usr/bin/perl. It won't break any of the OSses scripts.

That still constitutes installing your own perl, right? The issue of whether it's in /usr/bin or elsewhere is relatively minor, to me. It's smart of them to tie their scripts to a specific installation somewhere outside of the regular hierarchy (and that's what I recommend we do with our programs and our own installation).