Devel::Declare vexes me greatly.
On one hand, it's interesting to see the kinds of things you can do when you have the ability to temporarily take over parsing of the language.
On the other hand, this ability to just parse whatever the hell you want means the gradual decline in utility for the entire Perl development toolset. It's like someone has taken source filters (which everyone at least KNEW were "evil") and made them popular and "safe".
The real problem here is not that these problems are occurring, but they are occurring for what often seems like (or actually is) a very good reason. And as anyone who has seen my "Nothing Can Possibly Go Wrong" talk should know, the most dangerous failures are the ones we are seduced into because it seems like such a good idea.
For example, take this cool implementation of URLs via Devel::Declare.
Mixed that with LWP::Simple and you can do things like this, that actually run.
my $content = GET http://ali.as/;
Or how about inline SQL, as featured in a couple of other languages. I'm fairly certain that by adding hooks for, say, SELECT, INSERT, UPDATE and DELETE I could easily make the following work.
my $arrayref_of_hashes = SELECT * FROM table_name;
This stuff is supremely shiny, and if you're anything like me as soon as you look at it you want it badly.
But these features aren't free.
In the general case (looking at ARBITRARY modules rather than specific modules) you lose the following.
1. No/degraded syntax highlighting.
2. No/degraded PPI parsing.
3. No/degraded shiny Padre refactoring tools.
You also lose support for things like Perl::Critic almost entirely. PPI's statement classifier drives a lot of critic functionality. It only exists because PPI can reliably find the ends of statements.
If you break the ability to find the end of a statement (as many D:D modules do to define their own block keywords like "class { }") then the parse tree starts to get really really whacky and critic will inevitably start to spit out false positives.
Once you start spewing false positives in enough volume, you undermine confidence in the tool and we might as well stop using it (or at least turn off entirely rule subsets).
There are proposals for ways to extend PPI so somehow support these new keywords (I've even agreed with Matt on how we would go about it) but I continue to suspect they don't work in the general case because you have to have installed the PPI extensions in advance, before you attempt to parse a document containing those D:D modules (or you allow arbitrary execution of user code, at which point Acme::BadExample will complete pwn your machine).
This process gradually grinds onwards. D:D gets more mature and more used, Perl 5.12 contains core enhancements that help make it work better, further legitimising the concept.
Each step along the way is completely rational, and completely defensible. And it all looks great in blog code snippits. To paraphrase one prominent Perl core dev "Should we stop making the language better because it breaks PPI?"
And yet, at the risk of bringing down Godwin's Law on myself, I can help but see the kind of effect described in the famous poem First They Came.
First they came for the communists, and I did not speak outââ¬âbecause I was not a communist; Then they came for the trade unionists, and I did not speak outââ¬âbecause I was not a trade unionist; Then they came for the Jews, and I did not speak outââ¬âbecause I was not a Jew; Then they came for meââ¬âand there was no one left to speak out for me.
How many useful development tools are we willing to sacrifice (making development harder) in order to make development "easier".
I'd have some sympathy if there were any chance of getting (for example) function signatures, the method keyword, or the class keyword into Perl 5 this side of the heat death of the universe, but that's not going to happen because "that would make maintaining existing code harder".
Moderated -2, troll, wrong
mugwumpjism on 2009-12-17T04:06:06
You know, that's funny, because working method signatures in Perl 5 are one of my new favourite things.use MooseX::Method::Signatures;
method morning (Str $name) {
$self->say("Good morning ${name}!");
}Re:Moderated -2, troll, wrong
Alias on 2009-12-17T04:14:57
Which demonstrates my point that there will be an increasingly widespread use of source filters... (I'm assuming that's what that it implemented using).
No source filter required
schwern on 2009-12-17T04:48:22
Nope, MooseX::Method::Signatures is Devel::Declare and a heaping spoonful of other magic.
Re:No source filter required
Alias on 2009-12-17T05:14:41
... and Devel::Declare is a source filter, albeit (as peregrin put it on #p5p) "keyhole source filters".
Re:No source filter required
schwern on 2009-12-17T05:28:51
While its true DD and traditional source filters both change the source code, it is unfair to simply lump DD in with source filters. "Source filter" is a dirty word. DD does not share the traditional problem with source filters, that of trying to parse Perl and their global nature. For example, a source filter might decide to alter code-like strings and regexes. DD won't be fooled by that. It gets around this problem by (ab)using Perl's own parser.
DD brings in its own bag of problems, but calling it a source filter seems dismissive.
Re:No source filter required
Alias on 2009-12-17T05:58:59
I agree that it does not share some of the side-effects, and the level of collateral damage is greatly reduced.
But it still makes arbitrary changes to the lexical structure of the language. From a static parsing point of view, THAT is the critical problem here.
The problems with collateral damage are only a problem from the point of view of an executing parser.
Re:Moderated -2, troll, wrong
chromatic on 2009-12-17T06:23:16
I meant in the Perl 5 core.
And now the primary interface to Moose is using D:D or something like it
...
Actually, MooseX::Declare is just another way to use Moose, but very surely not the primary.
Many people are using MooseX::Declare, and find it very nice and elegant, but many others (myself included) are not willing to use MooseX::Declare or Devel::Declare in production because it is not yet mature enough. Personally, my reasoning is that I feel it still exposes too much internals when it errors and makes for some difficult debugging. Of course when MX::Declare no longer has these issues and is more mature, I will no doubt start using it more, but I can assure you that the primary Moose interface is still plain old Perl and will remain to be for the forseeable future.
For the record, I agree with you,
- Stevan
Re:Actually No
chromatic on 2009-12-17T03:05:57
I have to agree with chromatic, the language itself cannot be a test bed for experimentation....
I agree, but that's not my main point here.
It is a mistake to canonize features before they're ready, unless you have some mechanism of indicating that they may change in subsequent releases. No one wants the chaos of frequent churn.
It's also a mistake, possibly more severe, to refuse to add features widely-requested and often reimplemented in subtly incompatible ways.
Having D::D available with core support is a good thing for experimentation. Yet it's also risky if that reduces the possibility of adding necessary, missing features. Part of that is, as Adam suggests, the fact of incompatibility across an ecosystem less tightly bound than the core. Part of that is the lack of good defaults for everyone but language adepts who know the proper incantations du jour. Part of that is that D::D can easily become a short-term safety valve to work around systemic process problems that may harm Perl 5 in the long term.
Re:Actually No
Alias on 2009-12-17T03:12:44
I've updated the post to remove the suggestion that the CURRENT Moose API is D:D, or that it is specific to just Moose
I seem to recall something Nat said in his "What's With Those Python Fucks Anyway?" rant that "circles should be square so they're easier to draw and measure".
While I sympathize the the folks that write tools to do static analysis of Perl code, it seems rather putting the cart before the horse to restrict language development to serve those tools. It should be the other way around. It would be nice if we had formal Perl 6 style grammars to make life easier for the toolsmiths, but I'm not holding my breath. Source filters and Devel::Declare are open to broad abuse, but what's remarkable is its use has been very restrained.
Re:Circles should be square
Alias on 2009-12-17T05:18:29
While I sympathize the the folks that write tools to do static analysis of Perl code, it seems rather putting the cart before the horse to restrict language development to serve those tools.
If you could create the best language in the universe, but it could only be written using Notepad (or an equivalent), would anyone actually use it?
Support for good toolchains appears to be correlative with the success of a language (although proving this, and then further proving causality, would be an interesting exercise).
Source filters and Devel::Declare are open to broad abuse, but what's remarkable is its use has been very restrained.
... for now. As the financial people like to say, past performance is not indicative of future results. Granted, the analogy is poor, but new ideas like this do often go through something of a ramp up period.
Re:Circles should be square
schwern on 2009-12-17T05:44:47
If you could create the best language in the universe, but it could only be written using Notepad (or an equivalent), would anyone actually use it?
Sadly, yes. A less panicky question would be to say if you could do without syntax highlighting and indentation, which is really all the Perl specific stuff I use. I'd miss indentation, but syntax highlighting I can live without.
That said, its not EITHER we have tools OR we get to play with the language. The tools have already had to reverse engineer the entire Perl 5 grammar. If a grammar altering pragma becomes popular they can add a few more widgets. This makes the tools a bit more complicated, but its not some impossible task or even an order of magnitude increase in difficulty.
The FUD here is that DD is going to spawn a wild collection of crazy new grammar tweaks everywhere. Didn't happen with source filters, I don't see it happening with DD. Largely what I see reaching production quality is A) real signatures and B) powerful blocks. Stuff like bareword URLs and inline SQL can break the grammar too many ways because of their ill-defined content.
Source filters and Devel::Declare are open to broad abuse, but what's remarkable is its use has been very restrained.
... for now. DUN DUN DUUUUNNNNN!!!
Perl 5 went through its source filter craze, discovered the problems with it, recoiled, spawned Perl 6 grammars, and then largely went dormant because its unsafe. DD might succeed in becoming stable enough to get past the source filter high water mark.
Considering this:
my $arrayref_of_hashes = SELECT * FROM table_name;
I think it gains me nothing. Nothing at all.
Considering this:
method morning (Str $name) {
$self->say("Good morning ${name}!");
}
If I can get that (amongst other things), I will cheerfully give up:
Different people value different things. I'd hate to give up the items on that list, but I'd much prefer to work with a programming language which looks like it's been created in my lifetime.
Re:Weighing Options
tsee on 2009-12-17T17:18:08
That's why D::D should only be a prototyping tool and eventually such features should be available from the core.
Note, however, that making the new syntax available with a "feature" invocation carries virtually the same problems as D::D. You never know from static analysis (and non-recursive) whether it's on or not. But at least this would mean it's blessed and to be considered part of the language.
Re:Weighing Options
Alias on 2009-12-18T00:48:35
Well, the difference with use feature is that at least in that case it's official, in the core, works the same on every operating system and every host, and doesn't require the parser to leave the document.
So it's possible to add the relevant special casing in, where in the generic DD case that is not true.
Re:Weighing Options
Aristotle on 2009-12-23T17:06:41
Not really. Consider Modern::Perl, common::sense, and their ilk. And besides the ones on CPAN, any codebase can contain its own equivalent. Sure, you can special-case
strict.Re:Weighing Options
perigrin on 2009-12-17T17:54:14
That example of MooseX::Declare like code is well intentioned but I (personally) think *horrific* for proving your point.
function morning (Str $name) {
say "Good Morning $name"
}This won't piss off the people who think you're pushing OO at them.
Re:Weighing Options
Ovid on 2009-12-18T01:17:15
Both can be made available, but don't forget that Perl doesn't really know the differences between functions and methods. That being said, I want a modern language and P5P is too conservative to provide that to me. Regrettably, they may be right.
