Incremental, backward-compatible change has its place. But sometimes it's good to be able to break your ties with the past and change how you look at things.
I've become convinced that we need to do this with CPAN.
Don't get me wrong, the CPAN is not only wonderful, it's one of Perl's best strengths. That doesn't mean we can't make it much, much better.
Ok, so what do I mean when I say CPAN? I'm referring to a lot of different things.
It would be a benefit to look not only at our own past, but the accomplishments of other people solving similar problems. There are plenty of packaging systems, library management technologies, etc. and in the great Perl tradition, we should steal as many of their good design ideas as possible.
Step one is to come up with some sort of spec. Step zero is research. What have we designed? What have others designed? What are our needs?
It's going to be an exciting ride.
I don’t consider it an unfortunate aspect of its history that CPAN is crufty. All successful large systems start as successful small systems – organic growth is directly correlated with success. In effect, all big systems have some measure of cruft, though some more so than others.
I am highly skeptical of a push to rebuild CPAN from scratch in an integrated fashion. There are a lot of moving parts involved, and each of them is complex in its own right, so it would take a very long time – and there’s still no guarantee that the new design will work.
Even if, against all odds, you manage to succeed, the new system will then have to compete with the existing CPAN (you weren’t going to propose that CPAN be abandoned as soon as it works, were you?) – much like the biggest competitor to Microsoft Office is older versions of Microsoft Office. You’ll have to overcome a lot of inertia.
To even dream of retiring CPAN one day, you’ll have to move across all the distributions, thousands of which have not been maintained in years – so such a move would cause a lot of unnecessary bit rot.
A better idea than rewriting from scratch is to keep improving CPAN one aspect at a time, moving things forward steadily without breaking anything. Refactoring, in a word. This isn’t just theory; it is already reality and has been for years. There is ongoing work on a better installer (two of them, in fact). Packaging, being that it’s not something that should be addressed by the CPAN itself, has had a lot of work where it should be done: depending on your system, you use PPMs, Debian packages, RPMs, Gentoo ebuilds or something else. search.cpan.org wasn’t around from the start. The CPAN Testers project is newer than that, and the CPAN Ratings system was introduced just two or three years ago (I forget how long it has been); AnnoCPAN is even newer. The source to most of these things is available at least on request.
If you want a better CPAN, seek out whichever of these projects strikes your fancy and lend a hand.
Re:Second system effect?
DAxelrod on 2006-08-07T01:58:01
I don’t consider it an unfortunate aspect of its history that CPAN is crufty.Neither do I. Any large system that hasn't grown organically is going to have deep flaws. All I'm interested in is taking a step back, looking at how we've grown, and maybe considering growth in a slightly different direction.
I am highly skeptical of a push to rebuild CPAN from scratch in an integrated fashion.I would be, too. That's not what I'm suggesting, nor planning to do, nor hoping anyone else will do. Especially not the integrated part. The Perl culture is very rarely compatible with a single way to do anything, and that's another one of its strengths.
You go on to mention a large number of wonderful projects that are making the CPAN better. These are exactly what motivated me to think about this.
If you want a better CPAN, seek out whichever of these projects strikes your fancy and lend a hand.This is precisely what I was planning. My decisions on how to lend a hand will be a result of my thinking.
Thank you for your reaction, though. Now that I've seen a (completely valid) interpretation of what I wrote that was different than what I meant, I have some further clarifying to do. Another entry will come soon.
(As a side note, it's freakin' awesome to see the smart people in this community who I've respected for years actually taking the time to read what I write. Even if they disgree with me. Especially if they disagree with me.)
it is 'the' primary, and mostly the only, reason people using perl have remained instead of migrating to ruby or other technologies. Perhaps the only thing that makes perl unique and special is CPAN. People too often forget this and stupidly think that over-engineering it, like p6, will be a revolution in newness and functionality. The road is littered with bodies of the puppy dogs of enthusiasm unleashed every year about this time after the conferences are over and are eager to tackle a 'big problem' only to find that, well...there's a reason some problems remain over time.
I've kept up with the 6PAN conversations and though I suspect I will only consider it a worthwhile conversation if P6 ever comes to pass...it suffers from many of the problems that have kept p6 from being released years ago...when you turn something like this into an exercise of academic wankitude the committee will kill it by talking it to death. And killing backwards compat...doubly, perhaps even trebly so.
Life is messy. So is CPAN.
Re:It isn't 'one of'
DAxelrod on 2006-08-07T02:15:48
Perhaps the only thing that makes perl unique and special is CPAN.I disagree. I think that the CPAN is a part of the larger culture surrounding Perl, and that this culture is the reason Perl is unique and successful. (I also happen to believe that the roots of this community lie partially in the design of the language itself, but that's a different musing.) The people continuing to work in, on, and around Perl are what keep it alive.
Wonderful, I hope I'll be able to tap your knowledge a little bit further, both for pointers to these conversations (I've found many already) and for your oppinions regarding them. Expect me to pepper you with questions at some point.I've kept up with the 6PAN conversations:) I suspect I will only consider it a worthwhile conversation if P6 ever comes to pass...That's one of the driving forces behind my ideas. Just about every other part of Perl is being reconsidered, why not the CPAN?
when you turn something like this into an exercise of academic wankitude the committee will kill it by talking it to deathThis is an extremely good point. One way to attempt to avoid academic wankitude is to figure out what problem you're actually trying to solve, rather than to endlessly say "Wouldn't it be cool if..."
And killing backwards compat...doubly, perhaps even trebly so.I don't see having to break backwards compatibility. I'd rather not completely rule out backwards compatibility, on the other hand. Given that Perl 6 is an opportunity to break some backwards compatibility, it's good to have all the incompatibility happen at the same time.
Life is messy. So is CPAN.And that's not something I'm interested in breaking.
Re:Back-compatibility is compulsory
DAxelrod on 2006-08-07T14:31:47
You can't avoid back-compatibility. You just can't.Well, the incompatible syntax of Perl 6 will be doing that anyway. The fact is, there will eventually be code on the CPAN that perl 5 cannot run.* If we do it very carefully, we can use this opportunity to do some really nice things for those Perl 6 modules.
But apart from things currently being brokenish (which we can blame more on CPANPLUS, Module::Build and Module::Install)Wouldn't it be nice to have an infrastructure where we could have all those without brokenishness? Part of it is submitting patches to those projects. Part of it may be thinking carefully about how everything is structured.
* Even with v6.pm, (which is another awesome project), I don't see how you could wrap all the P6 code so that it would look like P5 + v6.pm to perl 5. I could be wrong.
