The current CPAN toolchain is broken.
Not in a "but it works for me just fine" and "users should upgrade" world it isn't. Competent users are doing just fine. It's the incompetent users we have a problem with.
If you want a demonstration, get a computer with a fresh untouched install of a 5-year old Perl and try to install anything of note by just doing 'sudo -H cpan -i Module'. It will almost certainly crash.
And if we ever want anyone that doesn't know Perl (who are by definition not competent with Perl) to install a Perl-based application from CPAN, we need to fix that.
Ironically, most of the things that are broken can be fixed easily, it's just that they involve multiple toolchain components, and the toolchain authors aren't as effective as they could and should be be about co-ordination.
So I'll be trying to get some of these fixed over the next little while, with a view to having a CPAN toolchain that doesn't suck ready for 5.10.1, the first long-term production release of Perl 5.10.
The first step in fixing this is really to get my (adopted) house in order and try to make Module::Install play well with others a bit better.
One of the problems has been that Module::Install contains Module::AutoInstall. This piece of embedded mitochondrial functionality will allow your Makefile.PL to launch a CPAN client inside itself to fulfil dependencies, IF AND ONLY IF it is not already running underneath a CPAN client.
Except it often doesn't work, because Module::Install has no reliable way to tell if it is being run underneath a CPAN client or being run by hand.
The CPAN.pm-detection uses an unreliable lockfile, and the CPANPLUS detection uses an environment variable PERL5_CPANPLUS_IS_RUNNING that lies. It doesn't actually mean that CPANPLUS is running, it just means that CPANPLUS is loaded.
Inside of Module::Install, I've been working on a new experimental Module::Install::With package that contains all the functions used for determining the environment Module::Install is running in.
Now this finally has a reliable way of detecting CPANPLUS. As of the most recent devel release of CPANPLUS, it now will additionally set the environment version PERL5_CPANPLUS_IS_EXECUTING to be the rel2abs path of the Makefile.PL it is running.
In this way, Module::Install can now detect not only that CPANPLUS is running, but that it is running ME and isn't around for some other reason. The next developer release will also (for future-proofing sake) add PERL5_CPANPLUS_IS_VERSION so that Module::Install can know what version of CPANPLUS is running, in case it has to implement some sort of workaround far in the future.
Unfortunately, CPAN isn't quite up with the game yet.
Due to that whole "not communicating enough" problem, the latest developer release of CPAN now for some strange reason sets PERL5_CPANPLUS_IS_RUNNING when calling Makefile.PL... which just won't do at all.
But before the next full CPAN release we should have that fixed, and it will instead set PERL5_CPAN_IS_EXECUTING and PERL5_CPAN_IS_VERSION in a similar way.
If you want a demonstration, get a computer with a fresh untouched install of a 5-year old Perl...
I'm not sure how to make it easy for people who don't upgrade their software to upgrade their software, when the defining characteristic of the group of people who don't upgrade their software is that they don't upgrade their software.
There's a point at which helping people do stupid things goes beyond negligence and on toward malice. Supporting ancient versions of Perl gets dangerously close to that line--else why in the world are we continuing to develop Perl?
Re:The Star Trek: Generations Fallacy
Alias on 2007-03-07T06:06:23
You make it easy for people that don't upgrade their software to upgrade their software by making the software help/force them to upgrade their software.
The only challenge is in doing it in a way that isn't stupid.
In essence, if the past is stupid and broken and will hurt them, and they don't realise it, then we drag them into the future unless they actively tell us not to.Re:The Star Trek: Generations Fallacy
chromatic on 2007-03-07T06:56:18
Is your plan then to produce a minimal amount of software that can upgrade an ancient Perl installation into something modern, at which point the entire upgraded toolchain is available?
I'm all for that.
Crippling all of the nice modern tools that take advantage of all of the nice bugfixes and new features developed this millennium in Perl for the sake of people who refuse to upgrade, however, bothers me a little bit.
Re:The Star Trek: Generations Fallacy
Alias on 2007-03-07T10:58:30
My plan is to firstly make the toolchain aware of the fact it is fatally out of date, and to give it the ability to upgrade itself.
If, in the process of doing so, an ancient Perl discovers that part of the toolchain is not longer supported on that version of Perl, it should intentionally stop and tell the user CPAN no longer is supported on Perl version $blah, rather than just crashing.Re:The Star Trek: Generations Fallacy
chromatic on 2007-03-07T18:26:32
Provided that only a minimal amount of code has to be backwards-compatible with, say, 5.004_x, that sounds reasonable.
Re:The Star Trek: Generations Fallacy
Alias on 2007-03-07T18:36:20
That should be the case.
From my monitoring of heavily used modules, CPAN as a whole has had a fairly clear dependency on 5.005 for a long time.
And there's been a trend over the previous year to 5.6.1 as a minimum.
Plus of course the 5.8.5 unicode sanity point.
We'll see how it goes, but I don't expect the requirements to be too onerous, and the amount of code should be fairly minimal.Re:The Star Trek: Generations Fallacy
link on 2007-03-07T09:57:33
A 5 year old perl install is 5.8.0 (well 4 years 8 months). Is that really old enough to be called ancient?Re:The Star Trek: Generations Fallacy
Alias on 2007-03-07T10:02:28
Yep, go try it.
Typically what will happen is that some module (say, Catalyst) will have a dependency that uses Module::Build in the Build.PL. But dependencies don't get installed until AFTER Build.PL.
Circular unresolvable dependency. Explode.
Unless you first upgrade Bundle::CPAN, the standard CPAN client doesn't work.
And anyone that isn't a Perl programmer is going to have no idea that they have to upgrade CPAN to unbreak it before they run it.
Re:The Star Trek: Generations Fallacy
Alias on 2007-03-07T12:51:27
Sorry, I didn't realise you were replying to chromatic, not me.Re:The Star Trek: Generations Fallacy
chromatic on 2007-03-07T18:25:46
There have been eight stable releases of Perl since 5.8.0. That number will soon be nine. It's ancient.
And if we ever want anyone that doesn't know Perl (who are by definition not competent with Perl) to install a Perl-based application from CPAN, we need to fix that.
Is that really what we want to be aiming at? I mean, I think that CPAN (and CPANPLUS) is great, but the only people who know about it are Perl developers. It's great for us that the same method works (for some value of "works") on every platform that we use, but for the average user (or the average adminstrator of a computer with Perl installed) probably isn't a Perl developer and wants to install Perl and CPAN modules using whatever installation tools he is used to using to install the rest of his toolset.
I'd love to see a way to make it easier for CPAN authors (or perhaps CPAN itself - or another project attached to CPAN) to create packages that can be used to install CPAN modules using packaging mechanisms that are native to various operating systems - whether it's rpm, deb, sis or something else.
That's how you'll get people installing CPAN modules painlessly.
Re: Fixing CPAN: Part 1 - Toolchain Communication
Alias on 2007-03-07T12:44:01
Yes, sysadmins that don't know Perl should be able to install the latest and greatest off CPAN.
Yes, newbie programmers on the first day they start Perl should be able to install modules without having to deal with errors they don't yet know.
You are correct that it would be better for users to use binary packages, and solutions for these exist.
Unfortunately, nobody that I'm aware of has managed to find a way to package and maintain up to date a reasonably large subset of Perl. debian is at 3,000ish out of 11,000. I believe ports is higher, but I'm not sure of the current number.
Or what if they don't use an operating system with a good set of binary packages.
There's always ultimately going to be a need to move up and pull modules directly from the CPAN source repository.
Frankly, I'd rather they install modules painlessly regardless of the method they are using.Re: Fixing CPAN: Part 1 - Toolchain Communication
Aristotle on 2007-03-07T17:42:09
Is that really what we want to be aiming at?
Is that really what we want to not be aiming at? How does having a working CPAN toolchain preclude binary package repositories? Why should we not aim at having both?
Re: Fixing CPAN: Part 1 - Toolchain Communication
kane on 2007-03-08T14:31:07
I'd love to see a way to make it easier for CPAN authors (or perhaps CPAN itself - or another project attached to CPAN) to create packages that can be used to install CPAN modules using packaging mechanisms that are native to various operating systems - whether it's rpm, deb, sis or something else.Take a look at CPANPLUS::Dist::Deb and debian.pkgs.cpan.org and you'll see you and I are on the same page
:) It still needs some more hacking to make it easier to write new C::D::* modules, but the concept works
:) Re: Fixing CPAN: Part 1 - Toolchain Communication
Qiang on 2007-03-08T20:52:33
one gotcha i found with using debian CPAN repo is that the installation dir is different from CPAN installation. and CPAN is not aware of what module is installed by debian repo. since CPAN module path is ahead of debian CPAN module in @INC. it may introduce problem when you accidentially installed a newer version module that is already installed by debian repo. the faulty module is however hard to spot sometimes.
there is also dh-make-perl in debian. IIRC, cpan2dist works better than dh-make-perl.Re: Fixing CPAN: Part 1 - Toolchain Communication
davorg on 2007-03-09T09:23:32
Ooh. Very cool.
Now, of course, personally I don't care at all about Debian packages. Where is CPANPLUS::Dist::RPM.
Or have I just volunteered myself?
Perl packagers should install Bundle::CPAN with any new installation -- this is the approach Vanilla Perl takes and the result is quite sane for installing any module. It's reasonable to put this onus on packagers -- it's like ensuring that bugfix packages on any older piece of software are in place.
Don't use Autoinstall -- most of the gyrations you describe are just to fix the problems this creates. The module toolchain should leave it to CPAN/CPANPLUS/etc to resolve dependencies. Makefile.PL just needs to communicate what those are. Autoinstall breaks the well-established system of divided responsibilities so it's no wonder problems ensue.
Take away Kwalitee if Autoinstall is used and let the problem work its way back out of the system.
Re:Two answers
Alias on 2007-03-07T12:49:18
Unfortunately, even if everyone did bundle Bundle::CPAN, that only solves the problems that have occurred in the past. It doesn't solve the problems that will occur in the future.
To most of the old Perl distributions out there that predate the invention of the new generation of toolchain modules, they all worked just fine at the time. It was the future that bit them.
And so we need to try to deal with the bugs that we don't know have happened yet in the toolchain.
As for AutoInstall, you are superficially correct. At this point I'm all for a Kwalitee decrement for using auto_install in a Makefile.PL.
HOWEVER, unfortunately Module::AutoInstall contains large portions of required functionality. Things like the implementation of feature() and anything that handles prompts to the user.
So some of the dangerous functionality gets loaded ANYWAY.:(
Add to the the problem of targetting both Module::Build and ExtUtils::MakeMaker and this encapsulation of the "playing well with others" code is really needed anyway in order to deal with the larger problems throughout the internals of Module::Install.
