In response to chromatic's latest maunderings, there's a reason that the Perl core heeds "the DarkPAN" -- it's the same reason that "the DarkPAN" is still written in Perl. Language implementors have no obligation to maintain backward compatibility, just like language users have no obligation to keep using a language. It's "insane" to think that a script written for Perl 5.6 will work with Perl 5.10, just like it was "insane" to think that a script written to run on HPUX would run on AIX, Solaris, BSD, etc. Perl is great because of its stability, and breaking that because... whatever it is you want... is insane.
I don't think chromatic is proposing that Perl start break back compat willy nilly. He's been quite clear that he wants to see Perl on a time-based release cycle with published deprecation schedules.
Imagine if Perl release 4 times a year. Then if there was a desire to deprecate something, a release could say "feature X will be removed after 6 more releases, 18 months from now".
That seems pretty reasonable.
Furthermore, no one out there is jumping from 5.6 to 5.10 without some serious testing. Even with Perl's (IMO) way-too-much back compat, there are plenty of gotchas in that jump, and you'd better do some serious testing first.
Straw men lack JFDI
nicholas on 2009-02-22T08:40:48
I don't think chromatic is proposing that Perl start break back compat willy nilly. He's been quite clear that he wants to see Perl on a time-based release cycle with published deprecation schedules.
Imagine if Perl release 4 times a year. Then if there was a desire to deprecate something, a release could say "feature X will be removed after 6 more releases, 18 months from now".
That seems pretty reasonable.
I don't disagree with anything you say. Heck, I agree with it.
But
I tried it. This is me. I have made more releases of Perl 5 than anyone using this site.
I tried making regular, 3 monthly, releases of Perl 5.8.x. As a volunteer, I couldn't sustain it. I see a lot of people saying "wouldn't it be nice", and
- none of them actually know what is involved as is,
- none have actually suggested ways to make it viable, such that people not as good, not as dedicated as me stand better than a snowball's chance in hell of survivial,
- none of them even ask "what could I do to help?"
and until there are people putting even one tenth of their money where their mouth is, please, everyone, do not be surprised if nothing changes.
(And no, this is not meant to be a personal criticism of anyone, let alone the person we all have to thank for DateTime, a most useful part of CPAN. It is merely that the most logical place to thread my opinion in, is in reply to his message)
Re:Straw men lack JFDI
autarch on 2009-02-22T15:04:06
Yeah, clearly the problem isn't (just) a lack of will. I'm not a Perl 5 core dev, so obviously I don't know all the details.
So why couldn't you sustain it? Is there some reason preparing a release is so much work? How do Gnome and Ubuntu and other projects do it? More people? More money?
One thing I have wondered is why we can't get a few paid people working on Perl 5. You'd think that there are enough companies out there depending on it that they'd be willing to kick in $10,000 per year each to TPF, and TPF in turn could hire a couple people.
I don't know if that's been pursued and it failed, or if it's obvious it won't work, or what.
Re:Straw men lack JFDI
reneeb on 2009-02-22T16:57:10
Yeah, clearly the problem isn't (just) a lack of will. I'm not a Perl 5 core dev, so obviously I don't know all the details.
So why couldn't you sustain it? Is there some reason preparing a release is so much work?
There was a long thread on p5p about regular releases for perl. I think this message is a good point to start reading (if I remember correctly): http://markmail.org/message/w4zbguq4cjitfvcb
One thing I have wondered is why we can't get a few paid people working on Perl 5. You'd think that there are enough companies out there depending on it that they'd be willing to kick in $10,000 per year each to TPF, and TPF in turn could hire a couple people.
It would be great if companies would give some money to TPF. If you look at the donations page of the perlfoundation site, you'll see that there are not that many big donations (I know that booking.com gave $50,000 to TPF last year - thanks! But such donations are rare).
Nevertheless TPF gave a grant to Dave Mitchell for releasing Perl 5.10.1.Re:Straw men lack JFDI
chromatic on 2009-02-22T17:58:16
I don't think your list is fair, Nicholas. I (and a few other Parrot developers) have a decent idea what it takes to make regular releases. We have over two years of experience doing just that. Parrot's not in the same stage of its lifecycle as Perl 5, but we've managed to reach a place where releasing a new version is very little fuss, comparatively speaking.
There was a thread last year on p5p where several people volunteered to perform releases, provided that they could follow a brain-dead easy script to put everything in order. To my knowledge, no one who could put together that script took advantage of several willing volunteers. (For goodness' sake, though -- volunteers wrote a perldelta.)
I realize that the bleed/stable branching model makes all of this difficult, and I realize that documenting the release process is not at all fun, and I realize that this work has a lot of little details and that capturing it all is not easy, but other projects have done it, successfully.
You're still of course welcome to say that that's all too much talking and not enough action, but whatever happened to those volunteers? What was wrong with the plan that we all discussed?
Re:Straw men lack JFDI
nicholas on 2009-02-22T20:02:21
There was a thread last year on p5p where several people volunteered to perform releases, provided that they could follow a brain-dead easy script to put everything in order. To my knowledge, no one who could put together that script took advantage of several willing volunteers. (For goodness' sake, though -- volunteers wrote a perldelta.)
Ah yes. This was my reply to that. Several people, most of whom in their replies quoted my message that precisely what was needed was people who could follow the key part of your (wise) instructions for parrot:
6) Find half a dozen trustworthy people who know enough about the project to work through the checklist in step 2. Their most important skill is the ability to figure out when that checklist doesn't match reality and update the checklist. They need commit privileges. They do not need to be the second coming of Chip.
i.e. anything but something brain dead easy and scriptable. No-one volunteered to do the hard stuff, particularly those people who had mis-volunteered themselves earlier, which is evident from the replies to that message of mine.
And, yes, true, that volunteers wrote all of the perldelta. But only about half of the perldelta you see was from the perldelta project. The volunteers there, human like the rest of us, ran out of steam, with some chunks not finished. Schwern found a heck of a lot of errors in the module versions. I fixed a heck of a lot of copy editing, wrote the New Tests section from scratch, and quite a lot of other things that someone else could have done, had someone else existed.
But in the end, something else you said then rings true:
Volunteers help, but we've discovered in Parrot that putting final responsibility on one person (no more than once or twice a year) is the only reliable way to get something done on time.
Re:Straw men lack JFDI
chromatic on 2009-02-22T22:03:08
I agree with all of that, with one caveat:
Releases aren't that big a deal. There's a new release on the schedule. This one doesn't have to be perfect. It just has to be better than the previous one.
We could discuss strategies for making releases much easier (getting rid of the blead/stable branches and making all features and integrations take place on git trees, getting smokes of those alternate trees, and pulling only when they're clean), but the first step is "Write down what it takes to make a release" and the second step is "Ask a couple of smart other people to go through that list and see what's wrong or difficult."
Only then do I think you should worry about the further steps. I think several volunteers are still waiting to do their part at step two.
Re:Straw men lack JFDI
educated_foo on 2009-02-27T05:55:13
I don't think your list is fair, Nicholas. I (and a few other Parrot developers) have a decent idea what it takes to make regular releases.
And it is a major reason that I no longer follow Parrot or Perl 6. It's too much trouble to keep up with the constant API and language changes. I'm often busy doing something else for a month or a year, and if I have to spend time fixing the working code you broke, I'll just do something else.
Re:Straw men lack JFDI
chromatic on 2009-02-27T08:16:29
And it is a major reason that I no longer follow Parrot or Perl 6.
That has nothing to do with regular releases and everything to do with the fact that no one has ever written Parrot or a full Perl 6 compiler before. If anyone could sit down and design completely such a system before writing any code, we'd have done so. Unfortunately, that's never possible for any system worth writing.
Re:Straw men lack JFDI
educated_foo on 2009-02-28T03:10:38
...and that "full Perl 6" keeps changing, and that Parrot internals (PBC, PMCs, IMCC, etc.) do as well. If Perl 5 had regular bug-fixing and feature-adding releases, but kept its current level of stability and backward compatibility, that would be great. Regular Parrot-style "break-stuff" releases... not so much.
Parrot/Rakudo/Perl6 is pre-alpha software. There's nothing wrong with that -- everything was pre-alpha at some point, and indeed pre-alpha projects can be fun -- but Parrot's being in that state places certain demands on developers, and makes it hard to build upon. The stability of things like libc, POSIX, and Perl 5 makes them easy to build upon, and making them unstable in the name of "modernity" undermines that value.
Re:Straw men lack JFDI
demerphq on 2009-03-22T13:14:41
I've been watching how some other projects manage this kind of stuff, and I have a few thoughts.
First, you were doing maint releases, and backporting from dev into maint. I dont think that many projects have high frequency maint releases, in most projects maint releases are instead "just enough for the release to not be a problem" and only made as absolutely necessary. If a project has high frequency releases its at the bleading edge not in maint.
Second, IMO backporting from dev to maint AT ALL only makes sense if dev and maint are relatively in sync. The further ahead dev gets, the more likely that you have to major work to back port a patch.
Now it seems to me that doing regular dev releases is MUCH less of a burden that doing regular maint releases. I would hypothesize as well that regular dev releases would make it eaiser for maint to stay synced themselves. As smaller dev releases would mean more major versions, and more major versions would mean that the bleading edge would tend to be closer in terms of code to its predecessors, which in turn should reduce the back port burden of important fixes.
So I guess while I am not going to argue with you about the burden of our current processes, its not clear to me that it isn't a self fulfilling prophecy. There would be no reason to do heavy work in a maint track if the dev track was moving faster.
Personally Im inclined to think we could accelerate our dev releases a LOT. In fact I think one is long overdue already. Although I know perfectly well how busy Rafael is and that rolling a release right now is probably not something he really wants to do I am somewhat of the opinion that we should make sure that we have all the major platforms passing test for a few days and announce 5.11.1 as soon as we can. And then do our best to announce 5.11.2 soon after. Who cares if that means that we do a LOT of minor releases in the 5.11 line. At some point or another well decide its good enought to go out the door, and we wont be talking about maintaining 5.10 well be talking about maintaining 5.12.
Anyway, I guess my point is that making the caboose more efficient isnt going to make a train faster. Making the engine go faster will. And we
/can/ make the engine go much faster in dev if we choose. Pretty much as simply as saying "today is 5.11.1 day". Its a blead release, who cares if it breaks here and there. Thats what they are for. Would I be correct in thinking that leaving the quality of the latest blead aside that to release 5.11.1 is a matter of creating a tag, rolling and uploading a CPAN bundle and sending out an email?
Yves
Re:Straw men lack JFDI
nicholas on 2009-03-23T13:03:07
Answering the easy bit:
Would I be correct in thinking that leaving the quality of the latest blead aside that to release 5.11.1 is a matter of creating a tag, rolling and uploading a CPAN bundle and sending out an email?
No, not just that. What really distinguishes a release from a mere developer snapshot is that a release has a perldelta file - a summary of the changes between the previous release and this release.
(And what distinguishes a maintenance release from a developer release is that in a maintenance release we try very hard not to have anything regress, whereas a developer release is "please try this, and tell us what breaks")
Re:Straw man
Ovid on 2009-02-23T08:45:29
One problem with this approach as being a "viable" upgrade cycle for deprecations is that you and I might upgrade regularly, incrementally and pay lots of attention to point releases, but businesses generally don't. They stay at 5.6.1 for many years and then gird themselves for a jump to 5.8.8. Thus, your nifty "four releases a year" means nothing to them. They don't dare upgrade their perl in parallel with those four releases (that would be very foolish of them), but they wait too long between releases (for the same reason they delay some developers paying back technical debt).
I'm not arguing for or against this behavior on the part of many businesses (but it's not as insane as people think), but it's pretty common behavior in most businesses I've either worked at or consulted at. "18 months from now" means nothing to them.
Re:Straw man
autarch on 2009-02-23T17:37:45
But those people will have the same problems with regular releases as they have now. Jumping from 5.6.1 to 5.8.8 (or 5.10.0) is a big deal with the current release system. If there were time-based releases, jumping across a huge number of releases will _still_ be a big deal.
In both cases, you'll have to test all your code very carefully. You may also just give up and not upgrade.
I've heard about Morgan Stanley's environment where the basically have apps deployed on every single version of Perl, and each app has its own set of modules.
That sounds crazy in some respects, but it's not all bad. New work can use new version of Perl and modules, and it's not tied to the requirements of old stuff.
Silly autarch! Urth is for adults!
educated_foo on 2009-02-27T05:48:06
Is "because fuck you a year from now" that much better than "because fuck you?" I like the way my Perl programs keep working when I copy them to a machine where the sysadmin has decided to install the "latest and greatest." The sysadmin likes the way I don't ask him to install an old Perl to un-break my working programs.
Re:Silly autarch! Urth is for adults!
autarch on 2009-02-27T14:31:26
So you expect the Perl core folks instead to do all the work for you, and ensure that when you do something that's likely to be unsafe, it's safe anyway.
That's ridiculous.
Re:Silly autarch! Urth is for adults!
chromatic on 2009-02-27T17:57:38
You missed "for free", "for longer than a decade", and "without seeing your code or even knowing that it exists."
*sigh*
educated_foo on 2009-02-28T02:59:44
The Perl core folks can do whatever they want (as if they need my permission!), but their emphasis on backward compatibility saves me a lot of headaches, and those saved headaches are part of why I keep using Perl. How hard is this to understand? I'm not making demands or threats; I'm just trying to explain why, in a world with too many programming languages, I continue to use Perl for day-to-day tasks.
Re:*sigh*
chromatic on 2009-02-28T03:28:44
[Their] emphasis on backward compatibility saves me a lot of headaches, and those saved headaches are part of why I keep using Perl.
That same relentless emphasis on backward compatibility creates a lot of headaches, too. How many security holes exist because two-arg
openstill works? Why do I have to write the same boilerplate every time I start a new Perl program to ask the compiler for help? Why can't we have aclasskeyword? Why is the return value ofsystem()inconsistent with the rest of Perl? Why do you still needmaketo install pure-Perl code in 2009? Why is XS so difficult to write correctly?I don't believe that sunk costs are a good reason to stick with a language. I prefer to use a language that makes my work easier and more pleasant overall. Perl 5 makes that difficult sometimes, because I know we could fix many of its flaws if we had the courage to do so.
(I do realize that not everyone sees the world the same way, but you don't have to upgrade.)
Re:*sigh*
educated_foo on 2009-03-04T02:49:24
How many security holes exist because two-arg open still works?
I have no idea. How many of my programs are easier to use because it still works? This seems like the same attitude that led the GNU folks to put in a non-STFU-able linker warning about using
gets. The C folks got it right when they madeccandlintseparate programs -- assume the user know what he is doing, but let him ask for hints.Why do I have to write the same boilerplate every time I start a new Perl program to ask the compiler for help?
Because you choose to do so, just as plenty of other people start their programs with "use Moose," "use My::Util," or whatever.
Why can't we have a class keyword?
I personally think it's silly, since "class Foo {" is equivalent to "{ package Foo;," but I doubt many people use "sub class(&)," so I doubt adding it would do much harm.
Why is the return value of system() inconsistent with the rest of Perl?
Because it returns a Unix exit code, where zero is success and everything else is a different kind of failure.
Why is XS so difficult to write correctly?
Mostly because the Perl VM is complicated. I have used both XS and Inline::C, and while the latter may be nice for simple stuff, there is not much difference when you try to make an idiomatic interface.
(I do realize that not everyone sees the world the same way, but you don't have to upgrade.)
Thankfully, I also don't have to sysadmin every machine I use. But this statement is telling: "Not everyone agrees with me, but those who don't, don't need newer versions of Perl." If you were Perl's sole author, this would be an off-putting but legitimate statement. But you're not.
Re:*sigh*
chromatic on 2009-03-04T08:25:56
If you were Perl's sole author, this would be an off-putting but legitimate statement.
My status as one of hundreds of contributors makes it no less a tautology: if you want absolutely nothing to change, don't upgrade. You can't have both.
Re:*sigh*
educated_foo on 2009-03-06T03:01:40
if you want absolutely nothing to change
Where did I say this?
don't upgrade.
Where did I say that I am sysadmin on all the machines I use?
I may be a dirty pinko commie about the real world, but I'm a software conservative. If Bob Sysadmin upgrades Perl to version X, and Jim-Bob Sysadmin keeps it at Y on a different machine, it's a waste of my time to maintain two versions of my scripts... er, programs. Perl's (past?) sometimes-painful focus on backward compatibility and portability is one of its main features.
We find for our fairly large install (175k SLOC application, with 50 CPAN direct CPAN dependencies) that about 80% of the work required to upgrade is the same, regardless of how many Perl releases we jump across.
