mod_perl Presence Dying at OSCON

hfb on 2004-06-08T05:21:00

stas writes "It appears that in the last few years we have had fewer and fewer mod_perl talks and tutorials at the big (non-YAPC) conferences. And that's a bad trend. It certainly affects the number of mod_perl job offers, since those who decide which technology to choose for their next project go to those big conferences and chances are very high that they will pick the technology that had a dominating presence in terms of tutorials and presentations."

"How can you change that? Go to the big (non-YAPC) conferences and especially to mod_perl tutorials. Since conference organizers get a lion's chunk of their revenues (even if only to cut it even) from the tutorials, that's where the battle is. The fewer tutorials on the mod_perl technology you have, the fewer mod_perl presentations and overall presence will be, the fewer new mod_perl jobs will be created.

This year mod_perl's presence at OSCON really sucks. We have two tutorials which are under a question that will happen at all and just one mod_perl presentation. Plus a few more related presentations of technologies riding mod_perl. Just a few years ago, we had about 25 tutorials and presentations. If some of the mod_perl tutorials will be canceled this year, don't expect the conference organisers giving them a second chance next year, which brings us pretty close to a zero presence. And that's a gloomy future if you ask me.

So if you want to change things, go to your bosses and ask them to send you to the tutorials. If they can't afford two, make it one and make that choice Geoffrey Young's "Programming the Apache Lifecycle." Why Geoffrey's? It's because the stuff I'm going to teach you at my tutorial is ready available in the online documentation and you can read it by yourself. But not so for Geoff's unique material. And all those who have ever heard Geoff presenting, will agree with me, that he is one of the best speakers OSCON (and other conferences) ever had and most of us, speakers, need to learn from Geoff. I'd go for Geoff's talk just for the great experience, even though I know most of the material he presents. His tutorial will be covering both mp1 and mp2 if you didn't know.

So if you enjoy your mod_perl job and want to continue so in the future, make sure that you come to the mod_perl tutorials and presentations at OSCON and other big conferences.

p.s. the deadline for tutorial cutoffs (at least 20 attendees) is June 21th. So if you were planning to go to the tutorials, make sure you register (or add the tutorials if you already did) before that date."


Uncertainty

jdavidb on 2004-06-08T13:42:58

I wonder if one thing holding people back on mod_perl is uncertainty as Apache and mod_perl move to version 2. I know that I personally don't even understand anymore which version of Apache is current and if version 2 is considered to be "released." I've been hearing about version 2 since 2001 or so, but it still seems to be the "development" version. And I hear vaguely that version 2 changes a lot of things, so this may have contributed to the fact that I haven't investigated mod_perl, yet. Perhaps many people are waiting until things settle down to invest in learning only the version of mod_perl that is going to exist long term. Perhaps some assurance as to the status of version 2 would help give confidence to those who, like me, are not in the loop enough to know what the issues are.

So, if I asked the question, "Which version of Apache/mod_perl should I be using?" what would the answer be?

Re:Uncertainty

jdavidb on 2004-06-08T13:48:08

Hey, how about a poll on the subject? How long has it been since use Perl; had a poll?

What's holding you back from getting into mod_perl development:

  • Uncertainty about which version of Apache and mod_perl is current (fear of learning a technology only to watch it change)
  • Unaware of what benefit mod_perl provides
  • Aware of mod_perl's benefits but my work is not web-based
  • Tried mod_perl and it was unable to figure it out on my own
  • I write my Apache extensions in C (or Parrot, or whatever)
  • I'm such a procrastinator I may not even answer this poll until tomorrow
  • I'm already a mod_perl wizard, you insensitive clod!

Re:Uncertainty

stas on 2004-06-08T15:05:09

We are almost there with 2.0.

I'm currently working on finalizing the API. That means - getting *all* of the API fully tested and documented. I think I'm about half way through.

Once this is done, there will be an API freeze. Then we need to fix a few remaining bugs (listed in todo/release file in the source distro) and then we will start releasing release candidates.

I really hope to get most of this done somewhere this summer. Of course if we get help we can accomplish that faster. But so far noone has offered such, that's why things are moving not as fast as we'd like them to. Documenting and testing some 400 methods is not a quick thing to do.

To answer your question: "Which version of Apache/mod_perl should I be using?" The answer is the sooner you start testing mod_perl 2.0, the sooner you will be able to move to it in production. That's because if there are any bugs, the sooner you report them, the sooner they will be fixed. And for the API changes, that should be over pretty soon.

Re:Uncertainty

sky on 2004-06-09T06:56:50

Sadly, there is still a big uncertainty what the entire point of apache 2, aside from furious wanking from the apache foundation, is. Yes, users of inferior operating systems will see a big speed improvement, but if you don't, there is no point as far as I can see.

Re:Uncertainty

hfb on 2004-06-09T10:48:06

Well, TPF has Perl6, the Apache Foundation has apache2/mod_perl2....but there are still p5 and apache/mod_perl to consider. Glass houses all around the village square.

Re:Uncertainty

sky on 2004-06-09T10:53:45

Indeed, I do however fail to see how this answer relates to my question. What is the point of apache 2? Please do not get into a point of what perl 6 is, because I cannot answer that.

Re:Uncertainty

hfb on 2004-06-09T11:42:28

Well, considering I'm in the camp of folks who don't really think there is an inherent meaning or point to life itself aside from the basics of life and reproduction, I'm probably not the best person to give an answer other than what folks give as a reason or justification for climbing Mt. Evererst: Because they can. If everything you did had a point to it, life would be rather dull.

Re:Uncertainty

sky on 2004-06-09T11:46:42

Surely, the point of life is to argue endlesslessssly in different forums?

Re:Uncertainty

hfb on 2004-06-09T16:26:28

Naw, it just keeps you occupied instead of bothering the rest of the planet who couldn't care less. :) Having a purpose doesn't usually imply a point.

mod_perl @ isp's

zatoichi on 2004-06-08T13:49:32

I wanted to learn mod_perl but finding a cheap hosting company for mod_perl is *nigh* impossible while PHP is ruling the roost there. All the ISP's that have Perl in a CGI configuration have PHP in a mod_php configuration. That needs to change but probably won't. All the posts I read is mod_perl is a bear to configure properly and so ISP's won't (or don't) do it. I could be wrong but my experience shows not.

Re:mod_perl @ isp's

stas on 2004-06-08T15:16:43

First of all, we maintain a database of mod_perl-friendly ISPs. New submissions and corrections of the outdated information is more than welcome.

Second, it's true that it's much harder to get ISP give you mod_perl account, due to the nature of mod_perl, mainly because users running on the same Apache host may abuse each others setups. I hope that Apache 2.0 will change that. There are several prototypes for perchild MPMs, which allow you to run several processes, each su-ed into a different user (ala suexec but no forking at request time), and having threads running inside each of those process. At the moment none of these prototypes fully working, so I can't tell whether this will be the panacea, but we shell see.

Currently I know of these two implementations (there could more): perchild mpm and muxmpm

Re:mod_perl @ isp's

ask on 2004-06-08T20:42:15

You can get a small virtual server with root access and all for $10-$20 a month ...

    - ask

Re:mod_perl @ isp's

sheriff_p on 2004-06-10T08:11:42

http://xrl.us/bytemark for example

Simple mod_perl

hondo77 on 2004-06-08T17:38:51

It may be that people are using mod_perl for the simplest of reasons (faster scripts and persistent database connections) so they feel they don't need more tutorials.

Competition

mattriffle on 2004-06-08T18:21:09

I think part of it just comes down to competition, really.

Geoffrey's tutorial, for example, is up against both Damian Conway and MJD, which can't be helping things. Then you have the RT tutorial, which as a mod_perl application is probably siphoning away potential attendees.

I'll be at the "Patterns In Perl" tutorial during that time frame. In a perfect world I'd like to see Geoffrey's tutorial -- it's just a case of too many tutorials, and only one of me.

-Matt

mod_perl's narrow niche

Mark Thomas on 2004-06-08T21:04:23

Low-end users are flocking to PHP, as most ISP's are equipped with mod_php and there's a short learning curve.

Users who want more of a full-blown application server or a framework for building a large-scale application like a CMS tend to consider Zope.

Unfortunately, mod_perl seems to be squeezed in the middle, and the particular slice seems to be getting smaller...

I'd love to see a Zope equivalent in Perl. The closest things I can think of are POE and Maypole. Both POE and Maypole are wonderful, but are not really analogous to Zope--they are a bit narrower in scope. And they're not as mature.

Zope?

perrin on 2004-06-09T15:55:02

Java is a much more serious competitor than Zope. No one has built anything the size of Ticketmaster.com in Zope.

Prefer SpeedyCGI to mod_perl

Etceteral on 2004-06-08T23:14:47

Although there is a further speed increase as a result of have the Perl forker running in the Apache process, I've found SpeedyCGI/PersistentPerl to be a much easier solution than mod_perl itself. It can be installed without mucking with the apache conf file, can exist w/o being root, works for non-CGI scripts (can be used from the command line), and is overall much simpler (for me at least) to grok.

I hope mod_perl continues its development and continues to be popular, but I think discussion of "mod_perl" should really be expanded to include "persistent perl environments", since there are several others out there...

Re:Prefer SpeedyCGI to mod_perl

merlyn on 2004-06-09T01:21:48

SpeedyCGI and PersistentPerl are replacements for only the part of mod_perl using Apache::Registry. But Apache::Registry is but a teeny tiny part of mod_perl. You owe it to yourself to explore the other 80% of mod_perl and see what you're missing. (Hint, it's not just the content phase!)

Getting to OSCON

althalus on 2004-06-10T23:46:48

For me, as with many I'm sure, the whole questions is whether I can *go* to the conference. If I wind up being able to go, then heck yeah, I'm all over the mod_perl tutorials, but I have to actually get there first.

Mod_perl SUCKS

egarland on 2004-06-17T18:14:01

I love mod_perl, but it's horrible. I love it because I know it well, it's very powerful. I've developed in it for years and therefor I have my own set of libraries and standard development methodologies that I have developed over the years that let me quickly write applications that are flexible and powerful. Unfortunatley that doesn't change the fact that for most people, mod_perl sucks.

As I recently wrote in a post to the mod_perl advocacy mailing list:

Mod_perl apps are too hard. Too hard to write, too hard debug, too hard to install, and too hard to package and redistribute. I don't know why and I don't know how to fix it but I'm affraid if we don't a huge potential market for mod_perl and for the skills of the people who know it (us) will be lost.
Apparently, mod_perl is loosing market share and losing it fast. This is wrong. Perl is a great language and mod_perl has the potential to be a great platform. I know why I use it but I can see why other people wouldn't. All the problems should be fixable if we can just figure out what they are.

Has anyone out there decided not to use mod_perl? Why? For those of us who use it, what have it's limitations been? What have you been unhappy with?

Does anyone have any ideas as to what is needed to fix mod_perl and make it as easy to develop in as perl is for command line programs?

Re:Mod_perl SUCKS

stas on 2004-07-09T05:36:35

Hmm, please pardon my blantness, but what have you done to make mod_perl rock?

Too many people talk, too few people do. Most of the time when I ask for help to improve mod_perl code/tests/docs on the mod_perl list I get no offers of help.

Re:Mod_perl SUCKS

egarland on 2004-07-09T13:42:44

Sorry for not helping in the past, I haven't really been in a position to. I'll try to do more in the future. I'm trying to figure out how.

Perl is really the only language I am good at. I can program in C but I have done it so infrequently it's a stretch for me. I've ended up doing a lot more C in the last year than I ever have before and I'm finding it slightly liberating. I might be switching from a perl-only guy to a perl/C guy. At this point, however, I'd probably get involved in the iThreads stuff before mod_perl since I think it's current implementation so horribly breaks what mod_perl is trying to do that it will make things much worse.

Have you ever thought of doing one perl interpreter for each mod_perl script? Is there something that already does that? It would help with the memory separation and allow for better support for using threads. I've never wanted to do that before, is it already an option?

Re:Mod_perl SUCKS

stas on 2004-07-09T18:25:03

Sorry for not helping in the past, I haven't really been in a position to. I'll try to do more in the future. I'm trying to figure out how. Perl is really the only language I am good at. I can program in C but I have done it so infrequently it's a stretch for me. I've ended up doing a lot more C in the last year than I ever have before and I'm finding it slightly liberating. I might be switching from a perl-only guy to a perl/C guy. At this point, however, I'd probably get involved in the iThreads stuff before mod_perl since I think it's current implementation so horribly breaks what mod_perl is trying to do that it will make things much worse.

You don't have to know C to make mod_perl better. there are plenty of other things that we need help with. e.g. docs and tests. Just sign up on the users and dev lists and see what needs to be done. It's not hard to figure out.

Have you ever thought of doing one perl interpreter for each mod_perl script? Is there something that already does that? It would help with the memory separation and allow for better support for using threads. I've never wanted to do that before, is it already an option?

You already can do that, the problem is the slow perl_clone. In any case if you wish to discuss this topic please start a thread on the users list. it's very inconvenient to use this forum and hardly anyone will see our discussion

a solution only a programmer could love.

adamshand on 2004-06-18T02:09:14

i am a system administrator, not a programmer, and experience has taught me to go as far out of my way as feasible to resist mod_perl being installed on any machines that i'm responsible for.

why?

* mod_perl is the only thing i've ever used which makes apache segfault.
* mod_perl is significantly harder to debug then cgi or even php.
* with mod_perl my developers can bring down the entire web server with some casually bad code, and *i* have to track it down, proove that it was them and make them fix their code. the whole while everyone thinks i'm bad at my job because the web server is down.
* for whatever reason mod_perl code that gets written seems to be more buggy and destructive then normal cgi or php code.
* strace'ing apache is not fun.
* having someone's cpan install break your web server is not fun.
* you can't reliably run mod_perl as an apache dso module (recompiling apache isn't fun).

now i understand that just because there are ways that mod_perl can be abused doesn't necessarily make mod_perl bad. I also understand that it's an incredibly powerful tool that can make lots of hard things easy and the impossible possible ... all in less time. however my experience with mod_perl, at three different companies, has all been similar, and all been bad.

make mod_perl as easy to install, as easy to debug and as stable as mod_php BY DEFAULT and maybe you'll win back the sysadmin's of the world and they'll start installing it again for developers. in the mean time i'd rather manage more servers then deal with mod_perl. :-(

flame away ...

Re:a solution only a programmer could love.

egarland on 2004-06-18T03:21:05

* mod_perl is the only thing i've ever used which makes apache segfault.

It's true it can, but shouldn't. Perl rarely segfaults unless it has bad .xs libraries behind it although I think there are still some issues with bad sort subroutines doing it. If perl segfaults, apache segfaults since with mod_perl they are the same process.

* mod_perl is significantly harder to debug then cgi or even php.

Yup. It can be.

* with mod_perl my developers can bring down the entire web server with some casually bad code, and *i* have to track it down, proove that it was them and make them fix their code. the whole while everyone thinks i'm bad at my job because the web server is down.

You can mitigate this the way java programmers sometimes do, by running mod_perl apps on a separate httpd instance and having the real web server reverse proxy to it. Complicated but effective.

* for whatever reason mod_perl code that gets written seems to be more buggy and destructive then normal cgi or php code.

CGI code runs outside the httpd process so anything it touches has no effect on httpd. Mod_perl runs inside it so it can do bad things to apache if you aren't paying attention. Also, since the program is just a subroutine, it's variables are persistant which can cause big, hard to track down bugs. It's best to limit a mod_perl app to a very small number of httpd's (2) unless it's well tested.

PHP, from what I understand, cleans up everything after a web page is done being served. I hear you can make PHP scripts stay in memory like mod_perl but it doesn't by default. This helps with keeping the bugs down and the behavior predictable.

* strace'ing apache is not fun.

Nope

* having someone's cpan install break your web server is not fun.

Nope

* you can't reliably run mod_perl as an apache dso module (recompiling apache isn't fun).

Really? I've only run it as a dso lately and I've had no stability issues. I used to build it into Apache back when I was an Apache admin but I don't anymore. I just use the standard RedHat mod_perl which is dso.

make mod_perl as easy to install, as easy to debug and as stable as mod_php BY DEFAULT and maybe you'll win back the sysadmin's of the world and they'll start installing it again for developers. in the mean time i'd rather manage more servers then deal with mod_perl.

That sounds like a resonable set of requirements.

Re:a solution only a programmer could love.

perrin on 2004-06-18T14:35:57

Hmm, sounds like you have a situtation that is a little bit too wild west. Maybe some better testing and configuration management practices would help. If you'd like suggestions on how to improve the safety of mod_perl, you're welcome to ask questions on the mailing list.

mod_perl is the only thing i've ever used which makes apache segfault.

Bugs in C code are what make apache segfault. That can be PHP extensions, Perl modules with XS code, C modules, etc. Segfault bugs in mod_perl itself are very rare at this point.

mod_perl is significantly harder to debug then cgi or even php.

This hasn't been my experience: you have the same tools available for debugging in CGI and mod_perl. If you want debugging tips, feel free to ask.

with mod_perl my developers can bring down the entire web server with some casually bad code, and *i* have to track it down, proove that it was them and make them fix their code. the whole while everyone thinks i'm bad at my job because the web server is down.

That's true, but no different from any other embedded interpreter system, like PHP or ASP. If you frequently have problems with irresponsible developers bringing down your server, I'd suggest either giving them separate servers (separate apache instances, not machines) or using FastCGI or PersistentPerl instead to limit the scope of the damage that bad code can do.

for whatever reason mod_perl code that gets written seems to be more buggy and destructive then normal cgi or php code.

It's obviously not inherently more buggy, but it is possible to cause server-wide problems, unlike CGI. Again, all embedded interpreter systems have the same problem.

having someone's cpan install break your web server is not fun.

This is really a configuration management and testing issue, and it's no different from when you're using CGI.

Re:a solution only a programmer could love.

adamshand on 2004-06-19T03:34:18

Hmm, sounds like you have a situtation that is a little bit too wild west. Maybe some better testing and configuration management practices would help. If you'd like suggestions on how to improve the safety of mod_perl, you're welcome to ask questions on the mailing list.

I totally agree, it is too wild west but I'm just the sysadmin not the manager, not the team lead. I encourage proper configuration management procedures at every place I work but you don't always have supportive managment and sometimes there are legitimate business reasons to keep things fast and loose.

I'm also fully aware that there is a lot I don't know about mod_perl and there are lots of tools and knowledge out there that could probably make my life with mod_perl barable and possibly even pleasant.

But if everytime you stick your foot into the pool you get burned, sooner or later you stop putting your foot into the pool.

That's true, but no different from any other embedded interpreter system, like PHP or ASP.

It may be true in theory but it doesn't seem to be in practice. I can't remember the last time PHP code crashed Apache, load spiked or I/O bound my server. I've dealt with all three of those things, in multiple different instances, due to mod_perl in the last month.

And I'm not particularly defending PHP, but from an administrative point of view it is tends to be significantly better behaved.

Thanks for the friendly responses, I'm really not trying to troll or be rude, just hoping that maybe I can provide some feedback on what it looks like from the admin side of the fence.

Adam.

mod_perl tutorials are now SOLD out

stas on 2004-07-09T05:47:33

As Geoff has just mentioned to me, both mod_perl tutorials at the upcoming OSCon2004 are not only not cancelled, but are now SOLD OUT ( geoff's and mine).

Thanks to everybody who has signed up. Hopefully this will be a good reason to accept more mod_perl talks/tutorials next year.