All the Perl that's Practical to Extract and Report

Web in Perl.. needs work

xsawyerx on 2009-02-09T18:05:06

disclaimer: this was written out of frustration and perhaps a note or two are wrong, but the general concept is right. hopefully not for long


So, yeah, we got Jifty and Catalyst and Maypold and CGI::Application and Titanium and plenty more. Great. But lets start to break that down:
  • Jifty doesn't have a clean view. You can't give it to a designer because there's a lot of coding logic in there.
  • CGI::Application's modes are confusing to someone coming from the MVC world
  • Titanium and Maypold are projects made by serious people, but wouldn't you rather use CGI::Application or Catalyst instead?
  • Catalyst - by far the most broad and extensible and thus dynamic out of them all - still lacks (out of purpose) much of what web programmers need


We haven't even started the lynch yet (though I'm expecting to experience one at the comments), we're just starting.

What does a web developer really need in a serious system? First of all, a way to CRUD data (Create Read Update Delete) online - usually because you're writing a small CMS engine for someone or you don't like working with SQL directly to enter huge blobs of text. Makes sense. None of these MVCs have it. And why should they? They are MVCs only, not CMS programs. But.. this really is the first thing (and usually, the most) I (and most web programmers) really need.

Catalyst itself does not provide a lot of things because it goes after the concept of Perl of write another extension to do what you need and then just mix them all together. I love that, I really do. I love that if I want to do some nifty socket stuff, I just use it (and if it's not there, I install it through CPAN) unlike in PHP where you have to COMPILE SOCKETS IN THE FSCKING BINARY OMG! Then again, perhaps that's there's a lot of CRUD options which none of them work well enough for me or anyone else I personally know. Perhaps me and the people I get to meet are all idiots, but.. perhaps not.

So, when we look for something to do CRUD. We have:
  • Catalyst::Controller::CRUD
  • Catalyst::Enzyme
  • CatalystX::CRUD
  • ... buncha others
What all of these (and the rest of the *CRUD* modules on CPAN) have in common is that they are either over documented - meaning there is so much documentation you don't even know where to begin and have to put aside a week or two or three to read and experiment - or under documented, which is usually the case. I still don't understand why developers figure that writing the name of the method and the purpose of it in one line is enough. "Sure, I get it now! It's so obvious!"
It's worth noting that many of them didn't work for me, and I have really tried. One even refused to work with a simple DBIC schema, but ONLY dbdesigner4 files. WTF?!

It all started when I wanted some CRUD. At first I started writing it. Then after showing my girlfriend a part of mjd's awesome lecture on Red Flags, I figured I shouldn't repeat myself, so I tried to write generic CRUD functions but stumbled across some inheritance issues with FormBuilder so I went over to #catalyst to ask for some advice. The advice given was to try other form building modules. While I checked FormFu (underdocumented IMHO, but seems like worth a try), I was tipped off on HTML::FormHandler which is basically a Moose (love Moose) wrapper to handle the form logic, and has a small (yet extremely ugly) form rendering option. This, funny enough, was the only JFDI I've tried yet with all of these (and I've tried over 10, no kidding). All the rest are more than 30 lines of code or 30 piles of documentation to understand, or better yet, a few one liners that tell you NOTHING. With HTML::FormHandler, I got a simple CRUD up with just a few lines, expressively and easily. Finally!

However, my life is still not easy. Since HTML::FormHandler is so simple it's missing a lot of things, and I'm pretty sure bugs are still lurking (as in any software - as Steven Little notes). Now I have to write my own abstraction layer in order to allow CRUD for every possible controller that I want without having to replicate the code + minor differences between them. Hopefully it will work. Perhaps I'll have to patch some things.

Why isn't there any standard module that JFDI?! The greatest language in the world, and we can't have a database <-> web simple and working interface. Woe is me.

A quick translation

autarch on 2009-02-09T20:10:45

"I need this one thing so everyone else must need it too. What kind of shitty framework doesn't include this thing?!"

For the record, I have no use for that thing whatsoever, and I'm glad Catalyst doesn't build it in. I'm also glad Catalyst has a plugin system that lets people put something on CPAN that _you_ can use if you want to.

Re:A quick translation

jk2addict on 2009-02-09T20:48:07

http://chrislaco.com/articles/mvc-marathon/

On the other hand. When people need to start projects and they're looking into which Framework to use, the posters issues and your response are some of the same reasons people stop choosing or looking at Perl.

I love Perl and the Cat folks. But sometimes, the " lets make no decisions or include no requirements by default because everyone wants a different thing" is off putting at best.

As sucky as we say RoR, or Django, or CakePHP are for being inflexible, they became more popular than us for a reason: They did a lot more for the new user by default than we do. The above post is partial proof of that.

Rant mode off. :-)

Re:A quick translation

chromatic on 2009-02-09T21:41:01

They did a lot more for the new user by default than we do.

I can't think of a better description of Perl 5's main problem in the past decade.

Mod parent up ...

ulrich.voss on 2009-02-26T09:41:14

Worst thing about not cring about new users, RoR/Merb 3.0 will offer more flexibility regarding the view/the ORM/etc.

So Catalyst may lose its one and only big plus.

Re:A quick translation

autarch on 2009-02-09T22:28:12

But is the fix to jam all this stuff in Catalyst?

Maybe the fix is to make a super duper CatalystX::CRUD thingy, market the hell out of it, and recommend it to the new users.

Meanwhile, people like me, who want to get shit done in our own way don't have to deal with ripping out 100k of scaffolding that the opinionated framework stuck in our app when we said "catalyst.pl MyApp".

I do agree, being newbie friendly is very important. But that's more a question of marketing and and having something available. That something doesn't have to the core tool.

You missed the point I was trying to make

xsawyerx on 2009-02-09T22:50:44

I was pretty sure I'd suffer some flames for it because Perl programmers (me included) do not like to hear bad things and whenever someone says "this is bad", it automatically becomes "no, you suck!" and in most cases it's correct but this isn't a simple dichotomous situation in which either the framework/language is wrong or the user/programmer is wrong.

I don't think Catalyst should have default scaffolding or even built-in scaffolding. I think the Catalyst programmers have much more important things to work on than scaffolding. Important to note that I wouldn't object that kind of thing if it were to turn up but I'm not suggesting that this is the way it should be.

Beyond that, I do not think that this specific thing that I need right now (and almost every time) is the only thing that is important and that it automatically refers to everyone. You're taking things out of proportion and by so you're missing the entire point.

My point was strictly (through a rather long rant) that a major component in other frameworks that are essential to a lot of people is missing here. Beyond that, that every project I tried on CPAN (albeit I haven't tried all - and tomorrow I'm dedicating a whole day to keep trying) does not cover it or does not do it right - and that it's frustrating. I also tried to note that we have a documentation problem, and that we as a community still haven't solved something that deters a lot of programmers from using Perl-based MVCs.

I was pretty sure I'd get some flames in the regular type of "you're wrong, you whiny little bitch", but I hoped it wouldn't come from people who can read what I write. I just needed to blow off some stream from trying to do something in the language that I love that is considered easy and simple and straight forward in other languages, that I don't enjoy using.

Re:You missed the point I was trying to make

autarch on 2009-02-09T23:26:33

You needed to blow off steam, but you are upset that some steam was blown back?

I agree, the docs for Catalyst need serious work. I think having a super slick scaffolding tool for Catalyst would be great, as long as it wasn't something I had to work _around_ as a Catalyst user.

The thing about scaffolding is that it's very cool for new users, and absolutely useless for many people doing "real" apps (for some value of real).

Re:You missed the point I was trying to make

xsawyerx on 2009-02-10T08:31:44

You needed to blow off steam, but you are upset that some steam was blown back?

Alright, fair enough. I apologize for that.

I actually have very little complaints toward the Catalyst docs. Usually when I need to ask something with Catalyst, the docs clear it up for me. My docs "complaint" was towards plugins and additions people made, unless that's what you meant, in which case you're right.

I think having a super slick scaffolding tool for Catalyst would be great, as long as it wasn't something I had to work _around_ as a Catalyst user.

The thing about scaffolding is that it's very cool for new users, and absolutely useless for many people doing "real" apps (for some value of real).

Obviously if scaffolding would be automatic and built it, that would definitely be annoying. Having to delete it every time (like I have to arrange the controller POD every time I create a new controller to be UNDER the __END__ tag, please), not to mention worrying about it polluting the Catalyst core - which should be as clean as possible.

I think it should come as a plugin, and there already are a few of those. There isn't a single unified one everyone is working on, thus all those that exist are not flexible enough, don't work or don't have any proper documentation. We obviously don't get a lot of room to complain 'cos it's all free, and it's all volunteer work. Yeah, I know it is and I'm thankful for everything I get in CPAN, but.. god I wish I didn't have to cringe every time I need to write someone a website (because they'll need some sort of CMS to edit data in the DB) because I would have to write 4 functions (C,R,U,D) for each table I have (2? 3? 8? 12?!) which results in 32 subroutines that have so much duplicate code before I can even begin working on the manipulation of data (which is the core point of the whole website).
/rant

Sorry, I guess I had some more stream in me

Re:You missed the point I was trying to make

autarch on 2009-02-10T13:52:54

Improving an existing plugin would make for a gret TPF grant.

Re:A quick translation

chromatic on 2009-02-09T23:15:22

I do agree, being newbie friendly is very important. But that's more a question of marketing and and having something available.

It's 2009. Why can't Perl have good defaults by now?

Re:A quick translation

autarch on 2009-02-09T23:24:27

I'm not sure what "good defaults" means. Scaffolding is not a good default if you don't need scaffolding.

And I was talking about Catalyst, not Perl. Yes, Perl (the core) could use better defaults. Catalyst is fine the way it is (well, it needs lots of changes but not wholesale addition of huge features for a small audience).

Re:A quick translation

chromatic on 2009-02-10T00:01:15

If Catalyst is the preferred web framework for new Perl projects in 2009, it either needs good defaults such that novices can pick it up and start to be productive without having to make several technical choices they're not necessarily qualified to make, or it needs a very clear explanation that it's not for novices.

It's perfectly fine to say "Catalyst is a platform for experts who can assemble their own frameworks for themselves." It's not acceptable to say "Catalyst is the right place to start for everyone."

Re:A quick translation

phaylon on 2009-02-10T14:38:52

Because nobody wrote them, or thought about them more than just "I need this now, can we add it?"

I've been writing much more docs (for other projects, that is) in the past than I am now. I wonder if the large number of "Your docs are crap! I _shouldn't_ have to work this hard to understand it!" has anything to do with it. Sure, people _shouldn't_ have to worry about some things. But why should I worry? I have no gain from it. I already have the software.

Besides, it is often forgotten that the integrating "Patches welcome" methodology just works. That's how DBIC, Catalyst, Moose and others got their large number of committers.

Another problem is that there just isn't anyone to blame. We can't blame the people for wanting easier entry, and we can't blame the people for not doing enough for others in their free time.

Re:A quick translation

chromatic on 2009-02-10T17:10:05

Besides, it is often forgotten that the integrating "Patches welcome" methodology just works.

I've written a few patches myself.

Re:A quick translation

phaylon on 2009-02-10T17:53:01

I didn't mean to imply anything else.

Re:A quick translation

chromatic on 2009-02-10T18:20:09

Oh, I know. Several of those patches have gone unapplied because they change the defaults in a backwards-incompatible way.

The reason we can't have good defaults in Perl 5 is because we've never had good defaults in Perl 5, and why should we make things easier for new people? It's a bad reason, but it's a pervasive one.

Make it modular, flexible just like Catalyst

oliver on 2009-02-12T13:30:47

I think one of the reasons there are so many CRUD systems for Catalyst, and none of them ever feels right, is because the CRUD system is almost an anti-Catalyst concept.

Catalyst as you say is all about choices, flexibility in application, etc. A CRUD system is (usually) a drop-in single "app" which works with ORM X and HTML templating system Y. So each CRUD system has a different X and Y. I could see CRUD working far better for, say, Jifty.

I am the author of one such Catalyst CRUD system, and I fully accept it "works for me" and we make great use of it at work. I also hope I pitched the docs right so others can use it too :-)

I have recently been wanting to completely rewrite this CRUD system, but it's an overwhelming task. I want to reach the same end point as you - something which is flexible and yet easy to use. This is Hard. Hey, if you are going to produce a lovely Moose CRUD layer I can stitch into my system between TT and DBIC that would be appreciated.

For info, my CRUD system is "the Javascript heavy one with the awkward name", called CatalystX::ListFramework::Builder - see http://tinyurl.com/listframework and http://dragonstaff.co.uk/oliverg/ for a demo.

Re:Make it modular, flexible just like Catalyst

xsawyerx on 2009-02-13T10:04:30

I think CatalystX::ListFramework::Builder was the one I haven't tried yet but wanted to. Usually when you start checking these things out, you get discouraged pretty quickly and don't even remember what you tried or didn't try.

It's definitely the prettiest, and I'm checking the tinyurl.com slides right now. I think I'll try it today. Thank you.

Usually I write my own CRUD because:

  1. I want it done fast
  2. I want to make some checks in the way

There's a major problem I think in this area because it's tricky. For example, in CatalystX::ListFramework::Builder, the ExtJS library does the rendering but also the form handling (create, add) so if it was

[DB] <-> [form handling] <-> [form rendering]

it has changed to

[DB] <-> [form handling + rendering]

and somewhere there you still have to fit the code for the controller methods to have the CRUD functions. If I could make a wish, I would wish on a generic CRUD layers model and thus be able to provide a standard glue for each piece, but the piece move around with the inclusion of AJAX, etc.

Re:Make it modular, flexible just like Catalyst

oliver on 2009-02-13T10:13:50

Yes by all means have a poke and feedback on LFB is welcome - but at the same time I completely understand why you would want to go a different way.

LFB, as a "first stab" at this type of application, was a good learning experience and highlighted to me much of why CRUD is Hard to do right. But LFB is a mess of design, it's true.

If you look at the last slide of my LPW talk, I'm basically saying the same as you - what we need is a generic CRUD layer, or really a set of layers which conform to an API. Then you could have different frontends (ExtJS, plain HTML, etc), different ORM plugins, and so on. That would be my ideal.