Mark Overmeer gave a talk at YAPC::Europe 2006 about cpan6, the design so far is the result of a collaboration between Mark and myself. The talk was generally well received, and during the conference we have heard many more peoples' concerns. The good news is that there were no new requirements that didn't fit cleanly in the design, in fact it gave some people lots of ideas. I think I can say that we have support for the general direction of things, and now we can open up the debate widely, and start implementing pieces.
I invite people to join either the pause6 mailing list (for infrastructure discussions) or the cpan6 tools list (for client-side installers and upload tools).
The earliest task will be looking at the big picture, and seeing which pieces are the low-hanging fruit that we can write tests for straight away. I'll start the ball rolling after we have a few subscriptions.
Re:Questions, Concerns, Complaints, oh my!
hfb on 2006-09-08T06:49:40
The CPAN is very large and very complex, and replacing it isn't going to be as simple as we might think.
Not true. The CPAN is rather small by today's standards and very simple. Clearly, you've not read the Zen paper describing the mechanics of it in very simple, straighforward terms. You are correct in supposing that it will not be simple to replace, but likely not for the right reasons. Anyone can build an ftp archive, put it on the net and open it for business. What makes CPAN work is not to be found in the software.
I honestly cannot think of anybody involved in the existing CPAN that is qualified to design a complete replacement for the CPAN
Uh, because they just...did it...instead of talking it over and taking 6+ years to design the ultimate archive? Over the years, I have repeated the message countless times since every idiot who feels like their 'brilliant' idea goes unheard or their itch doesn't get scratched is a deliberate snub from 'THEM' who run the archive who sit around with their thumb up their ass thinking up ways to frustrate the users: CPAN is simple in its mechanics, but the problems that do exist are either very hard to solve or inherent in perl itself. Beyond that lies the social problems which, for the most part, sort themselves out without intervention.
My concern for this project is that clearly little research was done into what has been done in the past (e.g. that we had already agreed to Audrey's request to make a 6pan repos on the existing CPAN) and given that I've been told it was due to some hostility directed at those who run CPAN, it has a high potential go the same way as P6 and P5 have. CPAN is about the only thing keeping a lot of users of perl around and introducing a P6-like CPAN archive redesign project will be the last nail in the coffin.
Re:Questions, Concerns, Complaints, oh my!
mugwumpjism on 2006-09-08T07:34:42
My concern for this project is that clearly little research was done into what has been done in the past (e.g. that we had already agreed to Audrey's request to make a 6pan repos on the existing CPAN)While this was news to us, and our research did not uncover this - nor did any of the people at YAPC::Europe mention this to us - the efforts need not conflict; I hope they can re-inforce one another.
CPAN is about the only thing keeping a lot of users of perl around and introducing a P6-like CPAN archive redesign project will be the last nail in the coffin.This is a confusing assertion. Why would users care if another system is being built to replace it, especially if we are extremely careful not to switch the old one off for several years? It will be giving them only more options.
Re:Questions, Concerns, Complaints, oh my!
hfb on 2006-09-08T10:31:29
This is a confusing assertion. Why would users care if another system is being built to replace it, especially if we are extremely careful not to switch the old one off for several years? It will be giving them only more options.
Because if the same sort of design and implementation limbo that now exists with P6 with P5 more or less moribund, is introduced to CPAN, it has the potential to finally push away those who are already on the fence. One being dead and one not being implemented with far too many scratches to itch. On the upside, the P6 effect will be nice in the idea that all the people who simply like to bitch will move from CPAN to CPAN6.
There need be no limbo.
mugwumpjism on 2006-09-08T11:47:59
Simpler approaches to the distribution problem are already under development, in competing projects such as 6pan. I wish them the best of luck and hope in turn to be able to be of benefit to them. Perhaps someone could follow-up with the link?
I feel a bit shouted down with the way these comments work.Re:There need be no limbo.
hfb on 2006-09-08T12:28:01
6PAN is a competing project? *sigh*In that it is complementary and independent
mugwumpjism on 2006-09-09T09:30:45
AIUI 6pan is just another area of CPAN. This is difficult, because for 6pan there appears to be basically no information about what is happening, apart from an old stagnant mailing list.Re:There need be no limbo.
tsee on 2006-09-08T16:15:15
Personally, I think that competing Perl repositories will only make matters worse. One of the reasons why CPAN is successful is that it is *the* place to turn to for Perl modules.
Other than that, I won't repeat Elaine's, Adam's and brian's points because I agree with them. I think the three of you disagree much less than Elaine stated anyway.
SteffenRe:Questions, Concerns, Complaints, oh my!
Alias on 2006-09-08T12:13:17
> What makes CPAN work is not to be found in the software.
And that is exactly what I mean.
"CPAN" as an FTP archive is a non-issue, the actual distribution part is the simplest and most trivial part.
What makes it successful is indeed due in big part to the social parts of the system, and those are the very things that a notional replacement design would have to take into account.
> Uh, because they just...did it...instead of talking it over and taking 6+ years to design the ultimate archive?
Who did? Various people did various parts, or single pieces.
What the public at large sees as "The CPAN" is the conglomerate of systems, not the FTP archive. Creating some notional replacement for all of that at once is what I'm suggesting nobody is qualified to do.Re:Questions, Concerns, Complaints, oh my!
mugwumpjism on 2006-09-08T07:28:19
I see a lot of detail on what your solution looks like, but not so much detail on things like the underlying reason for it's existance, use cases, constraints, assumptions, priorities, and so on and so forth.The "Emerging problems" section of the Global Design Document cover a lot of the shortcomings of the current CPAN. Also see the section "Projects" in the chapter "Pause6 Organization" for some more examples.
I would like concerns to be aired on the mailing list, and any amendments to the design may be submitted as patches; if they pass review, then they can be included in the design. Nothing is fixed except the fact that things will undoubtedly change.How do you plan to deal with these sorts of issues... and how willing are you to change the design (possibly significantly) in response to them. Because so far the impression from this side of the internet is of a grand design unveiled in one go, and a rallying cry to go build it, when only a relatively small number of people have seen or been involved in the design.The design is still open for discussion, it's just that the incubation period of design favours a more tight-knit approach. By sending you and Jarkko the papers early, we were inviting your participation in this stage; however we appreciate that it is difficult to do.
It really is a "damned if you do, damned if you don't" situation, we have people on one hand saying JFDI, and people on the other hand crying foul that we're undermining CPAN, or that we're building something that will undermine parallel efforts such as 6PAN. In fact we hope that such systems would like to use CPAN6 as a transport and file delivery mechanism.
Re:Questions, Concerns, Complaints, oh my!
Alias on 2006-09-08T12:21:58
The setup you have there makes it really hard...
I'm already on a billion mailing lists, unless you plan to have it go onto nntp.perl.org I really don't want to be on two more.
I don't know how I'm supposed to generate patches to a postscript file (why isn't everyhing just text or HTML) and in any case, I'm useless with patches, and so is Windows.
Not to mention I can't actually read the postscript file on this system.
OK, I may be bitching here, but if 90% of the computer users can't play nicely without project structure, it's limiting participation.
> By sending you and Jarkko the papers early, we were inviting your participation in this stage.
I was given a copy to basically shut me up:) And asked not to comment until now...
The JFDI element is from the "show us the code" population. There's nothing like a working demonstration to discover the design flaws.
As for undermining, I think my only concern there is that merely by using the names "CPAN6" and "PAUSE6" you present yourselves as a successor to the CPAN, when that is not necesarily the case, you are more of a CMAN (Comprehensive Meta Archive Network).
Re:Questions, Concerns, Complaints, oh my!
brian_d_foy on 2006-09-08T14:56:05
I agree with Adam. I'm not sure how you selected Jarkko and Adam to be the early reviewers of the paper, but it seems to me that Andreas might have something good to say, and that the current PAUSE admins might have had a lot of wisdom to add. My first inkling that anything was happening is when someone subscribed me to the CPAN6 mailing lists. I suppose that's a nice jesture, but no personal note accompanied the subscriptions, and I promptly unsubscribed. I don't do mailing lists. Put it on nntp.perl.org and then I can deal with it.
My initial reaction to the project, much like Adam's, is that I don't have time to deal with a new system, I don't want to be on more mailing lists, and that the design documents were much too large to be useful. The only thing we really need right now is a way to store distributions in a file system so that the kooky Perl 6 module selection (with author, version, and source) can work. I didn't see anything about that. Once we can store things, we can build things on top of that.
The design focuses mostly on module users, and is very unattractive to me as a module author and PAUSE admin. It sounds like a lot of work. CPAN works for me because I don't have to do a lot of work.
Furthermore, I don't suspect that most of the "security" additions will not actually help those people who are in environments that forbid CPAN use. In my work with Stonehenge I've talked to a lot of different people in many different companies and situations, and none of them had ever said anything about wanting those sorts of features. Sure, those features may be more like various linux things, but these people also are usually stuck with a vendor configuration for their operating system too and can't install external packages for linux either. Even if Tim Bunce signed all his DBI dists with his private key, the IT managers are still going to ask "Who the hell is Tom Bunce, who's selling the support contract, and who do we sue?" These are not issues technology will not solve.
Elaine mentioned that CPAN worked because Jarkko JFDI. I've been part of discussion for the local Python group about a CPyAN, and they get into the same design phase you've done. Notice that Python still doesn't have CPyAN.
I'd rather not see another design-by-committee project, especially if a lot of the work is going to happen in secret. Coming up with a prototype on your own and showing it to everyone is one thing, but planning the future of everyone's module use without getting everyone's input is a bit odd.We have been consulting with people.
mugwumpjism on 2006-09-09T09:28:20
I'm not sure how you selected Jarkko and Adam to be the early reviewers of the paper, but it seems to me that Andreas might have something good to say,Mark has been in contact with Andreas along the way.
... and that the current PAUSE admins might have had a lot of wisdom to add. We basically sent it to the people who asked for it. We have taken the initial design, subjected it to review and now we are asking the wider community for input. There's no point in releasing something before it is ready, it just spreads too much disinformation. Please consider it a matter of politeness of not giving you something until it was something like ready, rather than a young undeveloped brainfart.
Put it on nntp.perl.org and then I can deal with it.That is a shame. There was an offer to move it to 'perl6-cpan' on lists.cpan.org; but that's not right, either - this project is not specifically perl6 related, it just came from the requirements of perl6. So therefore it does not fit under the perl6- space. I hope once this impasse is resolved through early discussions that we can move it there and get your input, too.
The only thing we really need right now is a way to store distributions in a file system so that the kooky Perl 6 module selection (with author, version, and source) can work. I didn't see anything about that.Excellent, you are describing the cpan6 part in a nutshell, though we are also adding auditing and cryptography.
Once we can store things, we can build things on top of that.Yes, that is the idea - the core cpan6 network distributes releases, and this is a logical layer for tools like pause6 to provide CPAN- and PAUSE-like services atop of.
Send patches to the TeX source, not the PS
mugwumpjism on 2006-09-09T09:09:32
And if you do not wish to send patches, you don't need to. Just raise issues like you have done.
