chromatic has been discussing the idea of an "official" support policy for Perl 5.
Like a few other topics, we disagree on how long that should be :)
He seems to like a number of somewhere around a year, whereas currently I base my (loosely held) preference on what I think Perl needs to provide to be compatible with business usage (where far more Perl programming is done compared to the open source world) on an internal SAP survey that asked customer CTO and CIOs what their preferred upgrade rate was for ERP systems.
The survey reported that they preferred to upgrade their ERP system "Every 8-10 years, on a Saturday afternoon".
Clearly this is an idealised position, but Perl is indeed very stable and business friendly, and it does currently hold to somewhere around that rate.
For example, the CPAN toolchain (which is probably the biggest driver of back-compatibility) recently abandoned 5.005 (10 years old) and moved to a new Long Term Support target of Perl 5.6.2 (5.5 years old). In practical terms we also unofficially support 5.6.1 as well (8 years old last month), so anyone using 5.6.1 is probably ok as well.
The main caveat is that anything involving Unicode is going to have a much higher Perl 5.8.5 dependency (4.75 years).
If we consider 10 years to be too long to be a practical target, and anything less than a year to be too short to be a practical target, we have a reasonable bounding range within which to make a decision.
And for such an important decision, we need to find a robust decision making process.
While opinion, debate and blog posts at 10 paces can be great for making negative decisions (ruling out a particular option as bad) it's not always the best way to make positive decisions (picking the optimal solution).
We need to see if we can find a data-driven method. The CPAN is a good starting point, but insufficient. And the DarkPAN (All of the Perl code out there that isn't in the CPAN) is inherently unknowable.
But in recent years, with the growth in publically-accessible repositories as the default, we're seeing the formation of an new and interesting strata of the DarkPAN that I'm going to boldly term "The GreyPAN" (unless someone has already come up with a better term and I didn't notice).
In a recent post to the Ohloh Forums, I noted that Ohloh was reporting a far larger number of "lines of Perl" than I might have expected, and I was frustrated by not being able to find out exactly where all of this code was.
While a few million lines are concentrated in prominent non-CPAN projects like WebGUI and Twiki/FOSWiki, the vast majority are spread out in all sorts of different projects, doing the same utility tasks Perl does everywhere in the business world as well.
If this is the case, then it seems reasonable that we can take the GreyPAN as a analogue for the structure and consistency of the DarkPAN (or at least, for the region of the DarkPAN created over the last 5-10 years, when most of the Open Source code in the GreyPAN repositories were created.
And since we have tools for parsing useful information about Perl code (and doing Perl version dependency analysis in particular) it seems fairly reasonable (albeit computationally expensive) that we could do a metrics run on all of THAT Perl code and get a feel for how much damage we cause by moving the "minimum supported version of Perl" up to a newer version.
I really like the Ubuntu model of designating certain releases as long-term support releases.
If Perl were to release twice a year, then it could mark a release every 3 years or so as a long-term support release, and support it for maybe 5 years (the idea being you have 2 years from the _next_ LTS to upgrade).
Feel free to tweak those numbers as needed, but I think the general idea is sound. Supporting every release for 8-10 years (or 5 years) is nuts. Supporting no release for longer than a year seems problematic as well.
Re:How about the Ubuntu model?
chromatic on 2009-05-19T06:27:17
Number of FTEs at SAP AG: over 51,000.
Number of FTEs at Canonical, Ltd.: over 200.
Number of FTEs maintaining Perl: 0.
I should write in more detail what I'd like to see, but if I were in charge, I'd have yearly major releases. Perl 5.12 comes out in January 2010. There are three quarterly releases: Perl 5.12.1 in April, Perl 5.12.2 in July, and Perl 5.12.3 in October. p5p strongly encourages people to upgrade to the newest quarterly release, when possible. These quarterly releases only contain bugfixes. They do not contain updates to core modules (except for serious bugfixes in core-only modules).
If necessary -- for a security bug or a very serious corruption issue -- Perl 5.10 might receive another point release during the 5.12 series. After 5.14, there will be no further releases in the 5.10 series.
A deprecation announced in 5.12 will occur no sooner than 5.14.
There are more details, and a lot of this hinges on what people consider "support" -- especially considering that that "support" depends not just on what the 900+ people in AUTHORS are willing to donate, but on what the ~7400 CPAN authors collectively support.
I realize not everyone agrees, but the orders-of-magnitude difference in FTE numbers are difficult to ignore.
Re:How about the Ubuntu model?
Alias on 2009-05-19T09:19:10
The number of employees is largely irrelevant.
It's an issue of what the market and users demand, and what (as a result) we feel will make them happier and more productive. Hence "support"...
My point with the long timeline was that for the corporates, "Every 8-10 years, on a Saturday afternoon" is what they want. And it's these people that employ a whole lot of us, and in 5s and 10s and 20s at a time.
It's all well and good to deprecate lots of things quickly, and move in different directions each year. But if we make it harder for business to work with us, if companies end up spending half their development budget just trying to keep their half a million lines of Perl from being deprecated, and no capital expense can be amortised over a period longer than two years because it's going to bit rot too quickly and may need to be heavily rewritten, and if no feature that is added today can be trusted to stay around longer than a couple of years, then who in their right mind would risk using Perl?
Besides the ad=hoc scripting usage, one of the great reasons I've always been able to use for Perl/LAMP stacks against Microsoft stacks was that with Perl the language isn't going to be arbitrarily end-of-lifed every 2 years as something newer and shinier comes along.
So we may not give the CTOs everything they want. 10 years is clearly pushing too far, and even 8 seems to be towards the limits of what is reasonable.
But the number of FTEs certainly has nothing to do with it.
Re:How about the Ubuntu model?
chromatic on 2009-05-19T18:15:45
It's an issue of what the market and users demand....
If businesses want something they're not getting from the Perl development process, they should do what we expect individuals who want something from the Perl development process: contribute something more than their desires.
The number of employees is largely irrelevant.
I'll give you one reason SAP software is expensive: it costs a lot to maintain software for a decade. Where are those resources in the Perl 5 world? They do not exist now. Where will they come from?
I'd like to be a fly on the wall when you tell Nicholas or Dave or Rafael or Merijn that they need to multiply their workload because they're perfectly capable of maintaining several times the releases that they currently do.
It's all well and good to deprecate lots of things quickly, and move in different directions each year.
Are you arguing against something you thought I wrote? I never wrote that.
... if no feature that is added today can be trusted to stay around longer than a couple of years.... Who are you responding to here? Did anyone seriously suggest that?
... one of the great reasons I've always been able to use for Perl/LAMP stacks against Microsoft stacks was that with Perl the language isn't going to be arbitrarily end-of-lifed every 2 years as something newer and shinier comes along. Ditto this.
(One wonders how long knowledgeable Perl programmers will have to warn people not to use the Switch module, despite the fact that we've known it's been badly broken for several years, because we can't remove it until after a long -- unspecified in time -- deprecation process and because we can't suggest people upgrade to a modern release of Perl which supports a better version of the feature because it's not 2 pm on a Saturday in 2016 yet. That's... not exactly the way to produce the kind of reliable code I assume many businesses would like. Oh, and we can't predict if and when there will be a modern Perl release past 5.10.1 for these businesses not to upgrade to.)
I'll repeat my argument, as there seems to be a lot of distortion in the transatlantic cable today. The Perl community has the resources, if not the will, to release a new major version of Perl 5 every year. The Perl community has the resources, if not the will, to release quarterly minor releases of Perl 5. The Perl community -- mostly in the form of the CPAN -- has never demonstrated the ability nor the desire to support software for multiple years, for free.
Re:How about the Ubuntu model?
btilly on 2009-05-20T04:44:12
If businesses want something they're not getting from the Perl development process, they should do what we expect individuals who want something from the Perl development process: contribute something more than their desires.
It seems to me that businesses who are using Perl currently are getting what they want. Versions of Perl start having real trouble when they are 5-10 years old. In over a decade as a Perl programmer I have never been targeting a production system with a version of Perl that was under a year old. Only briefly have I dealt with one that was under 2 years old. I think I'm currently working with Perl 5.8.8. A few months ago I was targeting 5.8.6. And yes, I use CPAN. And have not encountered major issues.
From this perspective I see what you've said about rapid deprecation and just shake my head. As much as you say that the Perl community doesn't have the resources to maintain old versions of software, as a practical matter the Perl community does an excellent job of keeping old software working. You don't have to release new versions of old software every 15 minutes to support it. You just need to keep the infrastructure that software depends on from changing out from under its feet. And my experience of the Perl community over the last decade says that it does an excellent job of doing just that.
Re:How about the Ubuntu model?
chromatic on 2009-05-20T07:20:16
You don't have to release new versions of old software every 15 minutes to support it.
I disagree with your arguments, but I take them seriously. Why don't mine deserve the same respect?
Hyperbole, FFS!
educated_foo on 2009-05-21T05:49:38
s/minutes/weeks/ or s/minutes/months/. Now can you bother honestly addressing the rest of the post?
I suspect the real answer is "chromatic wants Perl to change quickly in XYZ ways; others want it to remain static in ABC ways; no one will convince anyone of anything." But this is just nerd-politics of the most pathetic sort.
Sarcasm is the First Refuge of the Incompetent
chromatic on 2009-05-21T07:02:58
Now can you bother honestly addressing the rest of the post?
What's the point? I've explained my point several times in several places. If people are still arguing against strawmen, no amount of logic on my part can possibly help.
Adam waves aside the point that sometimes work doesn't get done without people. I wonder how he explains that Perl 5 RCs do not get tested. (One rather suspects that SAP begs and pleads with very few of its 53,000+ employees to run through a test suite and a test matrix representing real world programs its paying customers rely on.)
Ben conveniently ignores that I believe quarterly releases are achievable, useful, and far better than what we have now to suggest that I want to remove features willy-nilly.
How do you recommend I respond to that? Sarcasm? Mockery? Yet another patient explanation that no, that's not what I believe and no, I've never suggested that, and if they read what I wrote and took it at face value they'd learn a lot more about what I believe, and meanwhile we've released 29 stable monthly versions of Parrot in a row and have increasing confidence in our roadmap for the next 24?
Releasing stable code on a predictable time frame is not impossible. Parrot does it. OpenBSD does it. Rakudo does it. The Linux kernel does it. Ubuntu does it. GNOME does it.
Tell me those projects haven't improved dramatically since the release of Perl 5.10.
Agreeing to disagree
educated_foo on 2009-05-21T21:25:49
If you believe you have already explained, then a polite "I think I addressed that here" with a link only takes the same amount of time as a non-responsive response.
You "believe quarterly releases are achievable, useful, and far better than what we have now," but as a Perl user I kind of like what we have. My scripts keep running, and from time to time the interpreter gets a bit faster, memory leaks go away, and new features appear. Yes, I am anxious to use given/when and some of the new regex features, but I can wait.
I don't actually use any of the projects you list, so I can't say whether or not they have improved dramatically. The only software I use with semi-regular releases is probably MacOS/Cocoa, which seems to take backward compatibility very seriously. It offers significant changes/enhancements every 12-18 months.
Finally, I'm not worried about stability: I agree that if you keep a checked-in code base well-polished and -tested, you can roll releases quickly and therefore regularly. I think it would be both useful and doable for Perl 5 to do this for minor releases. My only beef is with regularly scheduled deprecation, which is entirely orthogonal. It's appropriate for many projects, but I don't believe Perl is one of them. You clearly disagree, because (I think) you have a different idea of what Perl should be. Chacun a son gout.
Re:Sarcasm is the First Refuge of the Incompetent
nicholas on 2009-05-25T10:00:45
Gnome managed to screw up session management big time, and Ubuntu (and FC10) packaged it. Clockwork release cycles can bring failure as well as success.
Re:Sarcasm is the First Refuge of the Incompetent
chromatic on 2009-05-25T15:36:45
I debated putting GNOME in that list; I disagree with some of its philosophies (gconf is no substitute for a working configuration dialog, for example) -- but overall I do believe it's better after a few regular releases.
Software isn't perfect. That's why we have followon releases.
But while I agree we need to cater for business needs - I would not close any path at this starting stage - but rather think how to reconcile them with the business needs. I can see at least two strategies for that - one is a locked down system another is working with businesses on strategies for frequent updates (advising having a complete test suite, training stuff in upgrading etc).
I think your 8-10 years sounds about right. Perl is sort of like the OS or C library: many programs and modules depend on it, and they can confidently do so, just as they do C and e.g. POSIX. It's not necessary or practical to update all of them quarterly or yearly to compensate for changes. It's hard being backward compatible, but it's also hard building on shifting sands (e.g. Ruby 1.8 to 1.9, or 1.9 to 1.9 for that matter).
Re:i am confused by this discussion
chromatic on 2009-05-19T17:58:16
If you have brittle code that uses deprecated features, you probably aren't even going to consider installing the latest and greatest whenever it comes out.
Exactly.
Those people aren't going to upgrade until [they] HAVE to.
Exactly. Where are they getting support now? Not from p5p nor from the CPAN, and likely not from any core contributor. I completely fail to understand that mindset that it is the responsibility of anyone who wants to contribute a patch to the Perl core or upload a module to the CPAN to maintain code for a company which installed the most recent Perl the 20th century had to offer for free, without seeing the code.
Re:i am confused by this discussion
Alias on 2009-05-20T03:33:16
> Where are they getting support now?
In the case of where I'm working now (medium-large corporate) we don't get to choose our Perl version anyway.
Perl is an integral part of the underlying OS (RedHat) and while we may layer our own
/opt/cpan modules on top of that, we are bound by that. One of the recent bugs in RedHat that they fixed was due to us invoking our support contract with them and having them fix the bug, because we needed it.
I imagine a lot of other people are in the same situation. I'd have for standard practice to end up the way it's done on OS X, where you basically have to roll your own Perl to get anything at all done.
Re:i am confused by this discussion
chromatic on 2009-05-20T07:22:45
Perl is an integral part of the underlying OS ([Red Hat]) and while we may layer our own
/opt/cpan modules on top of that, we are bound by that. Red Hat's FTE: 2200.
Perl 5's FTE: still 0.
If Red Hat wants to support Perl 5.6.x, that's Red Hat's business. I still fail to see how that obligates p5p to support Perl 5.6.x.
Re:i am confused by this discussion
Alias on 2009-05-20T09:19:16
They aren't supporting 5.6, they're supporting 5.8.8.
But by your timeline there could easily be things in 5.8.8 that have been deprecated by now.
Re:i am confused by this discussion
nicholas on 2009-05-20T10:55:54
They aren't supporting 5.6, they're supporting 5.8.8.
Are they? I've see no communication in recent years from them about Perl 5.anything in RHEL. (As distinct from Fedora, where Redhat employee(s) working on it have been most communicative and helpful. I've no idea whether the same employees have Red hats as well as Fedora hats, in which case my question is probably unfair, as it could well be that by the time it comes to build a new RHEL from a Fedora, they've ironed out all their problems and simply don't need to ask any more questions.)
Re:i am confused by this discussion
Alias on 2009-05-22T10:46:51
> Are they?
Yes. Yes they are.
Re:i am confused by this discussion
nicholas on 2009-05-25T09:52:33
Good. Still, I wonder what they're doing, because it never ends up in feedback that I see.
Re:i am confused by this discussion
chromatic on 2009-05-20T18:47:11
But by your timeline there could easily be things in 5.8.8 that have been deprecated by now.
Yeah, wouldn't it be nice someday to have an XS that mere mortals can understand and use, with actual encapsulation such that perturbations in Perl's darkest guts don't break the code?
Hi Folks
To many people, lots of upgrades/releases suggest that work is being done.
But to other people, it implies the product is unstable, i.e. they view the frequency in a negative way.
Also, some people may be able to offer a huge amount of time, but for many of us the available time fluctuates.
So? Well, it's common for people to offer an unrealistic amount to support up front, and then find they cannot sustain that offer.
Such faulty offers are in fact normally but perhaps not always a symptom of a psychiatric condition, which would take a lot of effort to explain.
Lastly, businesses are often reluctant to upgrade at all, so we don't want to target a group who are not going to respond.
So, we need to be realistic above all else.
