Perl 5 Is Dying

Ovid on 2008-12-03T11:09:05

Take a look at the November 2008 TIOBE index top ten:

  Position   Position   Programming     Ratings   Delta Nov
  Nov 2008   Nov 2007   Language        Nov 2008  2007
  1          1          Java            20.299%   -0.24%
  2          2          C               15.276%   +1.31%
  3          4          C++             10.357%   +1.61%
  4          3          (Visual) Basic   9.270%   -0.96%
  5          5          PHP              8.940%   +0.25%
  6          7          Python           5.140%   +0.91%
  7          8          C#               4.026%   +0.11%
  8          11         Delphi           4.006%   +1.55%
  9          6          Perl             3.876%   -0.86%
  10         10         JavaScript       2.925%    0.00%

Yes, I know TIOBE has some issues. If you think it can be dismissed because of that, you're part of the problem.

Perl comes in after Delphi? We've still a ways to go before being knocked out of the top ten, but we have an *excellent* chance of this happening in the next year. Last year I seem to recall mentioning on the TPF steering committee list that Perl had been in the top ten of this list for years and was likely to remain that way. We were number six on the list and we weren't going anywhere.

I was wrong. Now I realize that there are those who think this is a non-issue and Tim Bunce has been doing a great job of fighting the FUD because he has the temerity to (gasp) use data. However, that's not enough.

Perl is in danger of falling out of TIOBE's Top Ten for the first time in its history and that's going to make news. And regardless of our biases, we all know of managers/programmers/venture capitalists who are happy to decide based on headlines rather than real data. Here at the BBC, it's no secret that there was serious talk amongst management and some programmers about completely eliminating Perl internally. We couldn't, partially for the same reason insurance companies and banks can't get rid of COBOL. The fact that Java is not a substitute for Perl has helped. The fact that Ruby on Rails doesn't scale has helped. When a scalable substitute with available developers (you Seaside fanatics put your hands down) crops up, Perl could be in trouble here -- aside from that pesky COBOL syndrome.

IE is facing stiff competition from Firefox because once Microsoft beat Netscape, they rested on their laurels. Hell, once Microsoft beat Apple and thought Apple would stay down, Microsoft stopped developing IE for the Mac! Java was facing stiff competition from C# because they rested on their laurels. C# being Microsoft-only is the only reason Java is still so strong, but even this has brought much needed changes to Java. Perl is facing stiff competition for the exact same reason that Microsoft and Sun are. Because we failed to innovate, we're now playing catch up -- just like IE and Java. 5.10, and perhaps 5.12, are too little, too late. I believe that outside of the COBOL syndrome, Perl 5 is in danger of having no serious long-term future.

When 5.12 hits we have a chance of winning programmers back, but the language has so much baggage -- both technical and social -- that I don't think it will win anyone back. We're losing because we didn't need marketing when we were the duct tape of the internet and now we sneer at it. We're losing because we rested on our laurels and scoffed at the up-and-comers. We're losing because older programmers ignored the concerns of the younger programmers. We're losing for the same reason that there are still people who object to images on Web pages, much less Flash and Javascript.

The technical world is evolving and we're fighting tooth-and-nail to not evolve (anyone remember seeing people sneer at the "REST fad"?). Perl 6 might save us. I don't think Perl 5 can, but remember, it's just a pile of code. What I'm really saying is that the Perl community as it currently exists cannot save Perl 5. We're reinventing the language. We need to reinvent the community.


Ruby on Rails...

fxn on 2008-12-03T11:31:49

"Ruby on Rails doesn't scale..." I guess that's implicitly mentioned as the kind of headline FUD some managers follow.

Re:Ruby on Rails...

Ovid on 2008-12-03T12:10:44

Perhaps it's more fair to say that we (the BBC) have suffered some scalability issues with it.

Re:Ruby on Rails...

boog on 2008-12-03T21:13:39

I attempted to pull my own teeth out once and made a right hash of it..

Ruby and Rails scale very well. The only issue is that it does require being on the ball regarding the latest tools and techniques, and to forget about how you'd scale a different language or framework - because the techniques mostly won't cross.

Re:Ruby on Rails...

chromatic on 2008-12-03T21:17:46

The only issue is that it does require being on the ball regarding the latest tools and techniques....

That's a good point. Which shiny new just-written web server does the Rails core team recommend this month?

... and to forget about how you'd scale a different language or framework - because the techniques mostly won't cross.

Rails doesn't have a shared-nothing architecture? Really?

Re:Ruby on Rails...

jplindstrom on 2008-12-04T14:47:15

Or even more fair, even accurate, to say that Rails didn't have anything to do with this particular case since it was an immature prototype that wasn't meant for production, much less being linked from the BBC front page.

And I don't know how a shared nothing architecture in any language "doesn't scale" (all the way up to the shared resource -- usually the db).

TIOBE Has Issues

chromatic on 2008-12-03T11:33:29

The presence of Delphi in the top ten should be enough to discount TIOBE's list completely. Delphi. More popular than JavaScript.

Perhaps there's some hidden Internet where Web 2.0 rounded corners are only possible through ADAX, and TIOBE's accidentally let on that such a beast exists.

Delphi, also known as Pascal with object extensions, from the days when Anders Hejlsberg was still at Borland -- a decade ago.

Re:TIOBE Has Issues

TeeJay on 2008-12-03T13:40:12

There really is no point even talkling about TI**E, the guy behind it is known to be partisan towards python, makes no secret of the fact he hates perl.

As chromatic pointed out, the fact he's included delphi shows how totally broken T***E is.

http://www.perlfoundation.org/perl5/index.cgi?usage_statistics

Shows plenty of metrics indicating that Perl community, usage, deployment and development is growing.

In fact all evidence shows it's being used more and for bigger projects.

Re:TIOBE Has Issues

zby on 2008-12-03T14:55:42

I went to that page and followed to the Perl job metrics - and in fact they were indicating that the number of Perl jobs is falling down.

Re:TIOBE Has Issues

TeeJay on 2008-12-03T15:30:46

Well Duh!

Seasonal fluctuations and the economy in London.

**Nearly ALL jobs are down**

That includes python, and especially ruby.

Re:TIOBE Has Issues

rozallin on 2008-12-03T21:26:48

To illustrate TeeJay's point, I've created a table showing the top ten TIOBE languages, the average hourly/annual rate of pay and their demand in the UK jobs market over the past year. IT jobs as a whole are down 47%, and at a glance every language is being impacted severely by the recession.

Re:TIOBE Has Issues

TeeJay on 2008-12-03T15:34:48

Oh look.. it's T****s Language of the Year 2007/2008 , what's it doing... ?

http://www.jobstats.co.uk/jobstats.d/Details.d/Trends.d/SKILL/PYTHON.d/stats.200 81202.1428.png

It's dropped by 50% since it's peak way back in 2006 and by more than 30% since it's 2007 peak!

TIOBE is definately on the money there then.. LOL

Re:TIOBE Has Issues

pudge on 2008-12-05T01:22:26

Let me say it more bluntly.

If you think TIOBE should not be completely dismissed because of its major flaws in both methodology and results, you're part of the problem.

TIOBE falls prey to the First Citiwide Change Bank fallacy: that just by getting a LOT of bad data, you can overcome the problems inherent in your methodology. "One word: volume."

But a lot of crap is still crap.

You Missed a Reason!

chromatic on 2008-12-03T11:45:25

It took five and a half years to go from Perl 5.8.0 to Perl 5.10.0 which, according to CPAN, is a "testing" release, whatever that is. Six and a half years after the release of Perl 5.8.0, there's still no "stable" new major version of Perl.

I disagree that gradual evolution has explaining power here. (Frankly, the only language above Perl on TIOBE's top ten list that shows even a flicker of evolution is C#. Python has a chance, but the modest goals of Python 3.0 may not compel upgrades. PHP's dying, and JavaScript was doomed the day Microsoft walked into the ECMAScript 4 meetings.)

Then again, some people still think that, in 2008, you should need to install Make and run cross-platform shell scripts to install Perl libraries written in 2008 but limited to features introduced in the shiny new Perl of 2000 (if we're lucky). I cannot rightly conceive of the state of mind which leads people to believe that idea.

Re:You Missed a Reason!

soulchild on 2008-12-03T12:37:15

IMHO, Ovid's points are absolutely spot-on and there's no need to play things down or look for excuses which might technically be correct but don't help the negative perception and reputation Perl has with managers and people new to the world of coding looking for a future-proof and appealing programming language. But most people in the Perl community don't realize this and will probably never realize it. If you look at how the world works you'll see that in most cases not the subjectively better thing wins but the one that's more shiny with better marketing (Beta Max vs. VHS, Windows vs. Linux, Perl vs. PHP, etc). Perl people say: "Yeah, but _I_ know that my technology is better so f**k the other's opinion." Try telling that your manager who has to make sure that there are enough developers out there for the technology he has to chose - which in case of Perl there aren't - at least not in Germany. Ruby and PHP have two big advantages over Perl: Charismatic persons or companies that push the languages (DHH, 37signals, Zend, IBM) and a community that has large quantities of designers who decided to start coding at one point. The former push the languages in the right places and the latter give them a friendly face. Perl has the advantage of having a community mostly consisting of highly trained software engineers who know how to crank out high quality code (which ex-designers aren't neccessarily capable of). But the problem is that most people still (and probably always will) judge books by their covers. Like many others I always cited the CPAN as one of Perl's killer "apps" but the majority of people doing web apps don't need over 40.000 modules. They need classes that provide the basic stuff (date/time handling, forms, db access, ORM, etc). And guess what: There's a large quantity of those classes for PHP, Python, Ruby, etc. so why stick to Perl if you can have a clean language like Ruby which has most of the libs you'll ever need and, more importantly, the "Good Feeling(tm)" that you're using/learning a language which is growing in popularity and which has a community of people that share the same problems you might encounter while cranking out your web app? Also, TIMTOWTDI can be one of the most frightening things when you're still in the learning stages (which might last a couple of years). So most people want something tried, tested and, most importantly, used by as many other people as possible, to increase the possibility of getting help if something goes wrong.

Re:You Missed a Reason!

soulchild on 2008-12-03T12:38:55

Err... I meant "objectively better", of course :)

Re:You Missed a Reason!

chromatic on 2008-12-03T18:31:28

If you look at how the world works you'll see that in most cases not the subjectively better thing wins but the one that's more shiny with better marketing....

That's not entirely wrong, but it's also too simplistic an answer. You're blatantly wrong about PHP's advantage over Perl; PHP had that advantage before Zend existed and before IBM cared. It's distribution and ease of beginning.

Strangely, that also explains the Rails advantage over... well, everything that didn't have a ten minute screencast at the time. Yes, that's partly marketing, but it's also partly technical. Ten years in, it's easy to argue that $4.95/month web hosts don't want mod_perl. The mod_php equivalent in the Perl world is...?

mod_perlite

srezic on 2008-12-03T20:22:09

It would be something like http://freshmeat.net/projects/mod_perlite/

Re:mod_perlite

jsut on 2008-12-04T15:39:54

Which doesn't really look like it went anywhere unfortunately.

Language versus Libraries versus deployment model

zby on 2008-12-03T12:47:43

I am not sure if the core language matters that much. Moose shows that you can have nice Object Orientation in Perl without changing the core language. Frankly I don't see much difference between Perl, Python and Ruby - for me the libraries matter much more. And let's look where Perl was losing. It did not lose to Ruby - but rather to Rails. PHP? Again, this already been covered somewhere, the advantage of PHP is the deployment model not the language itself. As I wrote in What can bring the excitement back to Perl?:

And what really makes the mentioned Drupal and RoR libraries exciting? With Drupal non-programmers can set up a quite complex community site on their own, without hiring expensive programmers. Paradoxically this creates also a big market for PHP programmers who later are hired to add new features to the already working community sites. With RoR it is the other way around - it let's a lonely programmer create a nice looking web application without hiring designers. This independence in both cases is quite a breakthrough and creates excitement and buzz.

I think Perl still have a chance. We need to shrug off some of the arrogance and dismissal, and accept that quick shiny prototypes (ala Rails) are important for the language marketing (and not only that - they are also practical). We need to accept that there are orders of magnitude more small websites that cannot afford all the costs of mod_perl than big web stars. But the potential is still there - it is in CPAN, unique in it's size and organisation.

What we need *now*...

nxadm on 2008-12-03T12:51:37

I would love to see the birth of Perl 6 as the next guy, but I think that a few additions to Perl 5 would make a love of people -including newcomers- very happy:

- We need a proper full blown Object Oriented framework. I don't care if it looks like java or ruby as long as it is standard (remember, we are catching up on this part).
If it's too hard to implement in perl core, deliver it as a Module (Moose should be a candidate to deliver something fast) and include it in the *Standard* distribution so everyone reading "Learning Perl" know how to construct standard objects. If someone wants to use bless to create his own OO world when tweaking for speed, so be it (TIMTWTDI), but there will be a professional OO framework for the rest of us.

- Proper error handling would be a bonus (try-catch-finally would be fine).

C.

Re:What we need *now*...

spx2 on 2008-12-03T13:46:29

This is not new nxadm , I have tried to report this here about 3 months ago http://perlmonks.org/?node_id=710860 . Frankly ... perl's built in OO is a joke and Moose is not far. Perl is old,it still has dollar signs attached to variables,it has weird built-in variables with weird meanings and weird behaviour.

Re:What we need *now*...

zby on 2008-12-03T13:54:53

I agree with most of your points - but sigils are useful aid when reading code.

Re:What we need *now*...

Ovid on 2008-12-03T13:55:23

Actually, quite a number of languages have sigils attached to variables. Function and variable names cannot clash in Perl. The weird "built-in variables" are an annoyance mainly because they have global scope. And I fail to see how "Moose" is a joke, though you're spot on about Perl's built-in OO.

Re:What we need *now*...

spx2 on 2008-12-03T14:12:43

there is no real book about Moose around,there are just some docs thrown around here and there and totally disorganised...it feels like an improvisation. I get that clumsy feeling whenever I approach it...

Talk about FUD

perigrin on 2008-12-03T15:24:53

In addition to the documentation we have now (which Dave Rolsky has a grant request in to improve), there was a talk at every major conference and workshop last year, there will be one at each of the next two workshops (Perl Oasis, Frozen Perl), and all of these notes and slides are available. There are over a hundred modules on CPAN that depend upon Moose, several thousand tests of various complexity (some show entire ORM sketches), and a hundred plus developers on the IRC channel who are willing to help out. So with all of that available, your argument is that you don't have a book to read so it must be a joke?

You do realize that several Perl authors have repeatedly pointed out that Technical Books don't pay, and that those of us in the Moose community who could (and would!) write a book have to feed our respective families and don't have the time or energy to devote to write. If you would like to pay me $60,000 USD dollars so I can exclusively work on a Moose book for the next year, I will happily do so. Also I'll need insurance for my family so make that $68,000.

Re:Talk about FUD

spx2 on 2008-12-04T04:20:05

In short : I don't care what your living costs are or what the costs you believe writing a book are.

Re:Talk about FUD

chromatic on 2008-12-04T05:36:43

Berating volunteers who've already done a tremendous amount of work, for free, for not doing more, for free, is rude.

Re:Talk about FUD

perigrin on 2008-12-04T17:20:02

In short: then you don't care about a book. Go jump in a lake.

Re:What we need *now*...

sigzero on 2008-12-03T14:44:45

I rather like the sigils. Makes reading the code a whole lot easier for me.

I am not sure it's dying...

sigzero on 2008-12-03T14:49:00

The thing I lament is when good tools become available for other languages and not for Perl itself.

As a recent example, there is a standalone IDE based on Netbeans for Python and/or you can code Ruby, Groovy, Python in the standard Netbeans IDE. No Perl. All the big IDE platforms support everything and their moma but not Perl (eclispe does have EPIC but...).

Check out or get involved in Padre then

TeeJay on 2008-12-03T14:54:18

The core perl community has a bias towards command line tools and more hacker oriented IDEs, vi and emacs are heavily used, but there are plenty of pretty good editors for Perl - Komodo (both free and commercial editions), not to mention scintilla based editors like Kephra and Padre.

The editors are there, but people don't need or rely on them.

Re:Check out or get involved in Padre then

Ovid on 2008-12-03T15:18:04

I suspect a lot of people couldn't name any IDE except Eclipse :)

Re:Check out or get involved in Padre then

perigrin on 2008-12-03T15:26:37

VisualStudio

Re:Check out or get involved in Padre then

Alias on 2008-12-04T15:47:00

At the present rate of development, Padre (and Chocolate) should have fixed your concerns by next YAPC::NA.

Lies, Damned Lies and Statistics....

draegtun on 2008-12-03T15:22:38

What TIOBE is measuring cannot be classified as a definitive "popularity of language". However it does provide a metric and so taking in context with other information it can be useful.

Of course that would only be true if the metrics were measured correctly! There are some obviously flaws in the approach they use...
http://www.tiobe.com/index.php/content/paperinfo/tpci/tpci_definition.htm

For eg. changing the rules to include +" scripting" could have a dramatic change to Perl (and other dynamic languages) figs ;-)

/I3az/

PS. And Ovid is correct there's more to this than TIOBE. Perl does have a PR issue that needs resolving.

PPS. Ruby fell out of top 10 but I wouldn't consider this language dead or dying.

Re:Lies, Damned Lies and Statistics....

perigrin on 2008-12-03T15:35:22

It doesn't matter if the methodology is flawed. The point really is that it's yet another sign that the rest of the world doesn't watch Tim Bunce's slide shows enough. Not when I had to listen to a co-worker at my new job say that Perl is dead and we should look at doing new development in Java. People who suspect something is true look for reinforcement of their ideas, that's why we all love Tim Bunce's slides and the rest of the world discounts them, and that's why we're all arguing that TIOBE is flawed or doesn't count, and the Perl is Dead crowd will crow about it if we slide off of the top 10.

Re:Lies, Damned Lies and Statistics....

draegtun on 2008-12-03T16:00:54

Well that's human nature I'm afraid. We all want to cherish whats important to us and attack anything that would diminish or threaten that belief. What we're actually seeing is other communities doing the same thing and having affect on the Perl community (very rarely will a Python post goes by without it slagging off Perl).

Well we can be resolute in our beliefs. But there comes a point where we do need to stand up and be counted for before its too late. Is this the time? I don't know but I'm happy to join any rallying call if need be!

Re:Lies, Damned Lies and Statistics....

Ovid on 2008-12-03T16:02:52

Or to put it another way: if the only people who hear that Perl isn't dead are Perl programmers, we lose.

Re:Lies, Damned Lies and Statistics....

Ovid on 2008-12-03T16:16:05

Better still: if the only people who hear that Perl isn't dead are Perl programmers, we're dead.

Re:Lies, Damned Lies and Statistics....

draegtun on 2008-12-03T16:20:59

Yep on both.

Time to prove we're not a Norwegian Blue Parrot (no pun intended!)

Re:Lies, Damned Lies and Statistics....

TeeJay on 2008-12-03T15:51:57

Not just flawed, it's known to be broken.

http://lui.arbingersys.com/ is actually vaguely useful. T*** is a waste of time.

There is no way that any dynamic language other than Python will ever do well on it.

Re:Lies, Damned Lies and Statistics....

draegtun on 2008-12-03T16:11:03

Hmmm... I wonder if Delphi rise as anything to do with it having "Chrome" in its grouping on TIOBE ;-)

And I wonder what would happen if CPAN was added to Perl's grouping??

Re:Lies, Damned Lies and Statistics....

perrin on 2008-12-03T17:13:21

Hey, that LUI one is really nice! Let's start using that one as a counterpoint to TIOBE wherever we see it.

Re:Lies, Damned Lies and Statistics....

TeeJay on 2008-12-03T18:01:36

or http://www.indeed.com/jobtrends?q=java%2C+c%2C+python%2C+perl%2C+ruby%2C+php&l=

or

http://www.jobstats.co.uk/jobstats.d/Details.d/Trends.d/SKILL/PYTHON.d/stats.200 81202.1428.png

You only have to take a look at this

inffcs00 on 2008-12-03T16:14:45

http://www.indeed.com/jobtrends?q=c%2C+java%2C+python%2C+perl%2C+ruby%2C+c%2B%2B %2C+c%23%2C+.net&l=

Re:You only have to take a look at this

Ovid on 2008-12-03T16:44:20

I find that very interesting, but that information needs to get outside of the Perl world and into the hands of decision makers. And just holding steady on jobs isn't enough. COBOL's done that, even though Perl looks better in comparison.

Of course you can put it in perspective

TeeJay on 2008-12-03T16:41:17

..by following the money... i.e. jobs..

Nice Graph that puts it PHP, Perl, Python and Ruby in perspective :
http://www.indeed.com/jobtrends?q=java%2C+c%2C+python%2C+perl%2C+ruby%2C+php&l=

Java and C are an order of magnitude bigger than Perl, Python and Ruby, Perl and PHP are considerably bigger than Python and Ruby.

Growth for all is slow but steady, no big drops, no big jumps. Nothing to see here... T**** is spewing random shit again.

DLJB (and the recession effect) reflects it

renodino on 2008-12-03T16:55:16

DLJB has been showing that the tech sector is not immune from the economic conditions, and that Perl seems to be taking a bigger hit than the other contenders (in pct decline). While the usual year-end slump is currently exaggerating the declines, the delta between Perl's losses (approaching 50%) seem more severe than Python, Ruby, PHP, et al. Only Javascript appears to be declining as sharply...so it maybe a Goliath effect. But still troubling.

On a personal note, I must admit my own interest in Perl is waning significantly. Sure, I'm still using Perl, but (I'm ashamed to say) my time and inclination for maintaining my CPAN modules, much less working on new CPAN goods, has dropped to near zero. Partly due to a new startup job focused on non-Perl stuff, but also due in part to a sense of beating my head against a wall trying to implement things other languages can take for granted (Groovy being my current playpen).

I also think CPAN is no longer the killer app it once was; Google is the new language-neutral CPAN (not code.google.com, per se, but just Google searches and cheap/free webpages available to developers to post their goods). Perl's community remains its strongest feature, but I sense numbers of users falling away from that as well (admittedly strictly a subjective observation, mostly due to perceived declines in perlmonk postings in recent months)

Perl 6 may save the day, but I suspect an odd effect of the recession may be to displace Perl with (PHP/Python/Ruby/Groovy/etc). Talent will be cheaper/more available, and maybe more inclined to develop in those languages.

Re:DLJB (and the recession effect) reflects it

jplindstrom on 2008-12-04T14:55:39

but also due in part to a sense of beating my head against a wall trying to implement things other languages can take for granted

I'd be interested to know what kind of things you refer to. For instance... ?

Tiobe lies

brian_d_foy on 2008-12-03T17:22:02

I think we can dismiss Tiobe, have proven it. Now you get to explain how I am part of the problem.

Tiobe is deliberately lying. They know what's wrong, they've intentionally distorted the data, and there's nothing anyone can do to fix that. It's not just flawed, it's inescapably unsalvagable. Making Perl higher in a list of lies, or even wanting to change Perl's position in a stack of lies, is itself a lie.

TIOBE's own comments are worth reading

drhyde on 2008-12-03T17:49:10

Further down their page, the people behind that rating list note that *Logo* just entered the top 20. I think that shows you how useful the data are as a measure of how alive a language is!

pages must contain "Perl programming"

chorny on 2008-12-03T18:55:22

Let's make sure that all our pages about Perl contain words "Perl programming", and maybe a link to http://www.perlfoundation.org/perl5/index.cgi?usage_statistics

Opinion from an apostate

mkc on 2008-12-03T22:39:38

Personally, I think the situation for Perl is worse than the TIOBE numbers suggest. I used to use Perl daily, but that's now unthinkable for me, considering the alternatives.

My advice: If Perl is to have a future, it's time for a radical revision. Specifically, all of the layers of old cruft that good Perl programmers don't use needs to be stricken from the language and all documentation. Barewords, be gone! This probably won't be sufficient, but I think that Perl is doomed without it...

Re:Opinion from an apostate

chromatic on 2008-12-03T22:47:42

That would be lovely, but it's never going to happen with Perl 5. It's all-but-impossible to add even a new warning on by default to Perl 5 these days, because (the horror!) code written in 1994 might start doing something a little bit different when run on a version of Perl released a decade and a half later.

I wish I were making this up.

Re:Opinion from an apostate

chorny on 2008-12-04T12:40:18

That would be lovely, but it's never going to happen with Perl 5. It's all-but-impossible to add even a new warning on by default to Perl 5 these days, because (the horror!) code written in 1994 might start doing something a little bit different when run on a version of Perl released a decade and a half later.

I once proposed to use "use 5.xx.xx" syntax for switching off old constructs, but it also was not welcomed.

Although some features where removed from Perl. $*, hashes as arrays, don't remember what else.

Perl 5.10, Moose, Devel::Declare, MooseX::Declare, Perl::Critic, PAR, Strawberry Perl, Portable Perl, configure_requires in META.yml, best practice by cpantesters - all this is a very big step forward. Unicode support is still best (compare this to horrible Unicode support in PHP - you need to use yet another set of functions) and in 5.12 it should be better. I hope they will upgrade....

Community

Mutant321 on 2008-12-03T23:15:25

Regardless of the problems with TIOBE, I think there is at least something of a downwards trend in Perl in certain envrionments.

There are a few reasons... not being able to find good Perl programmers is one reason given by my company. Also, wanting to standardise on languages as much as possible, especially for new stuff (where Java wins most of the time).

While I think the Perl community is one of our strengths, it's also a weakness. As you say, there are some people who refuse to acknowledge what's going on around them, as their way is the "only" way.

Having been working in Java for the last few months, I've seen a few of the advantages. Not really in the language, but in the tools and standards around it. IDE's are one that everyone focusses on. A lot of Perl people don't see the need for anything beyond vim, but the vast majority of people from a Java background (rightly or wrongly) won't touch anything without one.

Deployment and dependency management are also streets ahead in the Java world. You can build a .war file with Maven and copy it with all it's dependencies into tomcat, and it'll Just Work. You don't even need to worry about which platform you happen to be deploying to, which can be a big issue for some corps (yes, I know this is because of the JVM, but it's still a missing "feature" in Perl). I've managed to work around some of these problems using the likes of CPANPLUS, some scripting, and some third party build/deployment tools, but it's never going to be as robust.

Unfortunately, I'm quite pessimstic about this sort of thing ever improving in Perl 5. So big corps will continue to phase it out, and the jobs will continue to dry up. Although it'll be around a while, like you say, for COBOL-esque reasons.

Perl 6, and just as importantly, Parrot, are our only hopes.

Re:Community

mock on 2008-12-04T00:28:28

Not being able to find good Perl programmers is a huge problem. During the summer everyone I knew in the perl communities around Vancouver and Victoria was actively trying to hire people - with limited success. Things have slowed down a bit right now, but I'm told we'll be hiring perl programmers again in the spring, and that's the same story I'm hearing from most of the other startups around me.

There's certainly no lack of jobs from where I'm standing, and these are all software product development jobs, mostly for startups.

I think what is really lacking is marketing. Not a lot of us really talk that much about the things we're building with perl, probably because we're busier getting stuff done rather than trying to become the leaders of a new software movement.

So, to this end, a simple proposal: Those of us who occasionally put up hobby projects commit to having the "Powered by Perl" (from http://www.perlfoundation.org/perl_trademark) logo at the bottom of the page so folks can see for themselves all the cool stuff we're making.

Re:Community

dogcow on 2008-12-04T19:07:50

Lack of talent is a huge problem. Anyone know of a way to measure how many active job seekers list perl in their skillset. That would seem to be a better indicator than # of job postings with Perl as a requirement.

Re:Community

dogcow on 2008-12-04T00:42:21

To be honest I think the biggest major difference in perl vs java is not lack of an IDE (komodo is nice) but the fact that a mediocre programmer can generally write relatively clean, readable and maintainable java code. Why ? It forces good practices on you. Perl OTOH not only gives you enough rope to hang yourself, it ties it into a noose before handing it over. It takes someone with godlike levels of self control not to (even occasionally) take advantage of the many shortcuts around good programming practices Perl offers.

In the general sense though its I believe a declining technology for two main reasons.

1 - Web programming projects which would have been done in perl 10 years ago are now being done in PHP. Ease of use, clearer syntax, php.net documentation and many of the compiled in functions such as mysql support are probably the main reason for this. Not to mention the fact the casual user doesn't have to care much about mime headers, file modes and shebang lines.

2- Java and .NET have juggernaut companies backing them. Even if Java is technically an open project it is supported by a large co. Microsoft certainly offers support contracts and I assume Sun does as well. Big companies often times are concerned about accountability. The CEO/CTO wants to be sure some goofy compiler bug isn't going to hose their accounting system and if it does there is someone who can be taken to court over it.

Re:Community

chromatic on 2008-12-04T00:51:41

[A] mediocre programmer can generally write relatively clean, readable and maintainable java code. Why ? It forces good practices on you.

Precisely which features of Java promote maintainable Java code? Is there some magic JVM command-line switch which forces mediocre coders to choose proper identifiers, factor their entities appropriately, adopt the language of the business domain, respect Liskov, decouple unrelated entities, keep their methods small, and write precise and effective tests?

[PHP's] Ease of use, clearer syntax, php.net documentation and many of the compiled in functions such as mysql support are probably the main reason for this.

Clearer syntax? PHP.net's documentation? You're kidding, right? PHP's one real advantage is deployment. PHP.net is a vain defense against the bodge that is PHP's syntax and semantics (and I don't recommend reading the comments on PHP.net for anything other than amusement purposes).

Java and .NET have juggernaut companies backing them.... The CEO/CTO wants to be sure some goofy compiler bug isn't going to hose their accounting system and if it does there is someone who can be taken to court over it.

I can debunk that with two letters and the post increment operator.

Re:Community

dogcow on 2008-12-04T03:09:11

Precisely which features of Java promote maintainable Java code? Is there some magic JVM command-line switch which forces mediocre coders to choose proper identifiers, factor their entities appropriately, adopt the language of the business domain, respect Liskov, decouple unrelated entities, keep their methods small, and write precise and effective tests?
I didn't say it made them GOOD programmers but Java forces OO and makes you do things explicitly (strict typing, etc..). Perl's shortcuts and the level of things you can do implicitly lends itself to alot of cowboy type of code. We have all seen the type of code I'm talking about, and many times are guilty of writing it :-)

Clearer syntax? PHP.net's documentation? You're kidding, right?

PHP derives its syntax from perl and c/java. It has I believe largely very clear syntax , functions are called functions and not subs , it has a switch statement, predefined vars are not as ambiguous (I'm looking at you $_ ). PHP has come a long way since stuff like register_globals fiasco. They're OO is real and clearer than Perl's. I like the documentation, yes the comments are a joke but the documentation is clear and well laid out. Thats not to say perl is badly documented, I just think PHP's docs are a big part of its success. I also agree deployment.

I can debunk that with two letters and the post increment operator.

I assume this is a snark-y way to say C/C++ (if not I dont know what you mean). Obviously in your world Borland isn't a real company, and I guess MS never made any C compilers either.

Re:Community

chromatic on 2008-12-04T03:34:41

Java forces OO and makes you do things explicitly (strict typing, etc..).

You mean manifest typing, but yes, Java does. I believe that has very little to do with maintainability in the face of the other factors I listed. (There's also the counter principle that maintainability has an inverse relationship to the number of lines of code. Verbosity is not on Java's side here.)

PHP derives its syntax from perl and c/java.

Quick test: for any of the 1200+ global functions in PHP, tell me if the syntax is

find( needle, haystack )

or

find( haystack, needle )

. You get bonus points for remembering if and where underscores exist in the function's name.

Obviously in your world Borland isn't a real company, and I guess MS never made any C compilers either.

Microsoft has had a decade but hasn't managed to implement C99, so it's difficult to argue that they have a C compiler. Borland (apparently) makes its money from Delphi these days.

You're going to need lots of statistics to convince me that games studios pay Microsoft for DirectX support or embedded systems manufacturers pay Microsoft for Visual Studio support for C++. As far as I can tell, everyone who pays for MSDN pays for access to documentation, beta releases, example code, and not compilers.

Re:Community

chorny on 2008-12-04T12:31:25

I didn't say it made them GOOD programmers but Java forces OO and makes you do things explicitly (strict typing, etc..). Perl's shortcuts and the level of things you can do implicitly lends itself to alot of cowboy type of code. We have all seen the type of code I'm talking about, and many times are guilty of writing it :-)

Perl has very strict typing regarding scalar/array/hash. Types for objects are optional, although it is convenient, and I use it. For better perl code use Perl::Critic.

PHP derives its syntax from perl and c/java. It has I believe largely very clear syntax , functions are called functions and not subs , it has a switch statement, predefined vars are not as ambiguous (I'm looking at you $_ ). PHP has come a long way since stuff like register_globals fiasco. They're OO is real and clearer than Perl's.

Perl also has switch statement. It is just called given/when. Perl::Critic will also help with $_.

Re:Community

TheNomad on 2008-12-04T12:14:43

The argument that Perl's problem is that it allows too much flexibility is very common. But there has always been something wrong about it.

Surely, any decent programmer is going to want more flexibility. The more things a language can do the better, surely? Even if you don't use some features 90% of the time, the fact that they can be used 10% of the time can only be a good thing.

The problems that perl has at the moment seem mostly with mindset. Firstly of marketing - there is not one big high profile project at the moment that people say is perl. Catalyst doesn't have the profile of Rails. And nothing seems to have the profile of WordPress or Drupal.

Secondly, there's a bit of an Osborne effect with Perl6. I'm always thinking whether my code will run on Perl6 and what changes I'd have to make? What are the 20% edge cases that are going drain 80% of my time? How can I write perl 5 code that will make it easier for the converting parser? Will key packages like CGI.pm and DBI.pm work on perl6? The community needs to address this somehow. Sorry I don't have a solution just right now.

Thirdly, the backwards compatibility issues on CPAN that chromatic talks about need to be addressed. Simply put, module maintainers need to make the decision to break some old code. I guess I'm influenced by some XP practices which suggest that if you need to break old code, break it, but give people an upgrade path.

I guess all of these problems can be solved.

Re:Community

Lecar_red on 2008-12-04T21:19:37

To be honest I think the biggest major difference in perl vs java is not lack of an IDE (komodo is nice) but the fact that a mediocre programmer can generally write relatively clean, readable and maintainable java code.

I think the perception that this is true is the problem. And I don't know how to fix that. I have read lots (lots, lots, lots) of really poorly written Java code created with various IDEs. The IDEs allow people to discover language features that are hidden or require some research, but since they've been found it doesn't mean they understands how to use the feature (or use it properly without later bugs). I'm sure this is not a good thing for maintainabilty. I'm conflicted about tools, esp. ones that generate or help generate code. They seem to create lots of code that doesn't often quite work or later is maintainable (since they include confusing variable names and such). The big advantage of IDEs, as the poor maintainer later, is the support to easiy look up variable declarations and definitions.

As for your other points, I would agree that PHP has picked up many young and startup web coders. I think that is many due to it saturation in the web hosting space and its native-ish database integration. Also, for beginners its pretty easy to have a web template (from somewhere else), add php code then run through a web server and have a quick app/site. I think its pretty slick that it can be that quick. Also, its simple there aren't 10 choices for templating languages (well there are but not right away) or half dozen web app frameworks. I like the choice but not when I'm trying to figure out something for the first couple of times. In the beginning its just a lot of frustrating noise thats in my way.

Re:Community

pnu on 2008-12-04T20:15:41

Having been working in Java for the last few months, I've seen a few of the advantages. Not really in the language, but in the tools and standards around it. IDE's are one that everyone focusses on. A lot of Perl people don't see the need for anything beyond vim, but the vast majority of people from a Java background (rightly or wrongly) won't touch anything without one.

Deployment and dependency management are also streets ahead in the Java world. You can build a .war file with Maven and copy it with all it's dependencies into tomcat, and it'll Just Work. You don't even need to worry about which platform you happen to be deploying to, which can be a big issue for some corps (yes, I know this is because of the JVM, but it's still a missing "feature" in Perl). I've managed to work around some of these problems using the likes of CPANPLUS, some scripting, and some third party build/deployment tools, but it's never going to be as robust.

I don't really see this as the problem. Perl as a language / environment is more than "good enough" for most of the jobs it's been used for. Not everybody needs or wants to drive a Volvo, even if it's a good car.

For many uses Perl is actually a lot better than Volvo, people here know it, but it's quite a leap of faith for new users. Currently it's not too attractive, but that really is not a language problem. It is "just" marketing.

Please, we must do something to these '99 sites, and have this discussion outside!

Perl 5 is dying...

jrockway on 2008-12-04T03:29:22

... we should all stop using it!

Ovid is Dead On! Innovate or Die!

pjain001 on 2008-12-04T05:01:31

It doesn't matter whether you believe TIOBE or not. The fact is that new programmers are not even thinking about Perl. The only time they go look up Perl is if there's some "legacy" code in their jobs that's being thrown at them.

Ovid is dead on. We innovate or we die! I learned Perl in 1996. Guess what, not a whole lot has changed since then. I still write Perl b/c I love it but I don't do any more Web apps in Perl and I don't ask my guys to use Perl for web apps. Most of them go with PHP or Java/JSP for Web apps. Perl has been relegated back to being a command line scripting language for Unix geeks, nothing more.

I dread the day of having to learn another programming language b/c Perl 6 died on the vine and Perl 5 got stuck in a time loop.

Re:Ovid is Dead On! Innovate or Die!

pudge on 2008-12-05T01:24:46

The fact is that new programmers are not even thinking about Perl.

That is false. I have many new programmers ask me about Perl.

Companies: Teach developers Perl!

h1rschnase on 2008-12-05T12:08:57

When I was reading all this "Perl is dying" blogs in the last days, I was fairly shocked. WTF is going on with you guys? Winter Depression? Comparing Perl with Cobol and tell the world Perl is dying because some crappy Top10 I never heard about before say so? Perl is a really cool programing language, IMHO the most flexible one on the market. It's the standard tool for every serious Unix admin (and many Windows admins too) and I know many great Software Developer who're (still) in love with Perl - even if they're sometimes forced to use other languages in their jobs today.

My comment to the Perl Gurus at the BBC: I'm in a decision maker position in a big international Software Company. We're moving forward to use Perl more and more and we will use all the wonderful things like Catalyst, DBIx::Class, TT, Moose, mod_perl even MORE in the future for all of our front-end and back-end development. So, if you don't find any good Perl developers on the market, then go and hire people who're good developers in other languages (like Java, PHP, and so on) and TEACH them to become good Perl developers. Do INVEST in people instead of just try to benefit from what is already there!

A few years ago, I run my own Software Company and I was in the same situation at that time (yes, even in 1999/2000 it was hard to find Senior Perl Devs). What we did was that we hired junior devs and students who were willing to learn something new in a short time and then we did internal trainings with them. It's really not that hard to learn Perl for a OO developer - it usually takes a few learning sessions most of the time to get people started. If you're using standard frameworks like Catalyst the learning curve is very high and the results are exciting.

And yes, there are way more PHP or Java devs on the market. But, for example, try to find a Senior PHP Developer and you will fail. Try to find a really good Java Developer who is fast, flexible and has more background knowledge than what he had learned at the University; good luck.

What I'm saying is: Blogging about Perl and spreading the word is necessary, yes. But companies have the money and the power to change things by teaching people and make them become good Perl developers.

(Please excuse my bad english. It's not my mother language :))

Re:Companies: Teach developers Perl!

Ovid on 2008-12-05T12:28:18

Your English is fine. No worries :)

I do agree with your approach. We did this at one of my former companies and it worked fine. Subsequent employers have been very resistant because they assume they can get good developers out of the box. Still, I'm happy to hear that you and others are still encouraging ways to deal with some of the shortcomings we face. Thank you!

Please Volunteer for Thankless Tasks

Eric Wilhelm on 2008-12-05T19:52:57

As far as I can tell, there is little or no corporate funding or support for improving and maintaining our core infrastructure and key technologies. In the following list, I'm sure you can identify the one or two people who thankslessly keep the thing afloat. This basically means that the Bus Number of Perl is roughly 1, but if the loss of the maintainer of any one thing leads to failure, then the probabilities sum. This makes the bus (or babies or boredom) number more like 1/8.

* Perl 5 Core
* PAUSE
* CPAN
* Module::Build
* Extutils::MakeMaker
* PAR
* TAP::Harness
* wxPerl
* perl.org (sites and infrastructure)

Other than the issue of bus number, I'm also concerned about the fact that most of the important maintainers are volunteers who see little or no monetary reward from the efforts they put in. While the promise of better Perl jobs is nice and all, you can probably land a Perl job just as easily without taking-on such enormous responsibility. A vibrant job market is important to the health of the community, but several profitable, dedicated stakeholders (including universities) are necessary to invest in innovation and keep energy flowing.

And that's just to maintain our base technology and infrastructure. Things like community outreach and business development are similarly unbacked. For example, who will lead the Perl effort for Google's Summer of Code in 2009?

Re:Please Volunteer for Thankless Tasks

chromatic on 2008-12-06T09:16:28

Part of the problem is of our own making. Why in the world should anyone support MakeMaker anymore? It was broken in 2000. It's broken in 2008. We keep horrible code and awful designs around far too long (seriously, to customize MakeMaker, you write code which uses regular expressions to manipulate hopefully cross-platform shell scripts in the parent package from which MakeMaker's classes inherit?)

Why should we expect to recruit volunteers when we continue to perpetuate such horrors on the world? If I weren't so vehement about cleaning up our mess, I'd be embarrassed to admit that I know even that much about how such a thing works.

There -- now our bus number is 14% better.