I started thinking again about why class methods silently override role (traits) methods. Since I see this pop up on mailing lists from time to time by programmers getting hurt by this or a variant of this, I kept wondering why on earth the traits researchers would recommend this. From the original traits paper we have the following:
Methods defined in the class take precedence over trait methods. This allows the glue methods defined in a class to override methods with the same name provided by the used traits.
After pondering the Perl community debate on this topic, I still couldn't see the point. Sure, let the class win, but silently? That still doesn't make sense to me. Finally, knowing that any information I got from the Perl community would be second-hand, I figured the only way I was going to get an answer would be to email the original researchers and find out their reasoning. I confess that their responses held some surprises. While the following is quoted from Dr. Andrew P. Black, one of the primary researchers in this field, the others in the discussion generally appeared to agree with his explanations (the following quotes are all his and presented here with permission):
Yes, it is really important that a programmer can see clearly when a trait method is being overridden ââ¬â just as it is important that it is clear when an inherited method is being overridden.
So there you have it, right from the original researchers: class wins, but not silently. However, that raises the obvious issue of why this wasn't explicitly laid out in their papers? As it turns out, that's because they implemented this in Smalltalk.
In Smalltalk, where a program is viewed as a graph of objects, the obvious solution to this problem is to provide an adequate tool to show the programmer interesting properties of the program. The original traits browser did this for Smalltalk; the reason that we implemented it is that traits were really NOT a good idea (that is, they were not very usable or maintainable) without it. Since then, the same sort of "virtual protocols" have been built into the browser for other properties, like "overridden methods".
In short, the researchers felt traits would be a bad idea if the programmer couldn't explicitly see what was happening. However, this tool support is so integral to Smalltalk (you really should check it out), that adding behavior to the tools is second nature. The traits authors knew that this information needed to be immediately in front of the programmer, so they pushed it into their browser (kind of like the Smalltalk IDE) because that's as natural to them as a Perl programmer using map to transform a list.
When Smalltalk is in the browser, it's code, not just documents. Smalltalk, a dynamic language, gets around many of the parsing issues which Perl and other dynamic languages experience by having the code already compiled. You're not parsing a document, you're parsing the code. So what does this mean for languages where we parse documents? (Such a thing tends to be trivial in static languages like Java, but hard for dynamic languages):
If, on the other hand, you view a program as sheets of paper rather than objects, then the obvious solution is to add yet another language feature. So, Java, for example, realized (after about 10 years) that it needed an overrides annotation, and a compiler enhancement to issue a warning if a method overrides another without using the annotation. So they extended the language. Similarly, I agree that if the normal Perl environment does not provide good programming tools, then the implementation of traits for Perl ought to have done something to make it clear that a class method was overriding a trait method.
Make of that what you will. Personally, I don't use Perl::Critic and that's where the "notify when class wins" behavior was to be pushed (though I don't see it on the CPAN), so that's pretty useless to me. (Update: I phrased that very badly and I apologize to anyone I may have offended. See note below).
I don't know that this will necessarily cause movement on anyone's part, but it's nice to know I wasn't totally insane in my reasoning (though I originally wanted this to be a composition failure and was willing to compromise on a warning).
Do you also want using 'sub foo {...}' to override a superclass method to be a compile-time failure or warning?
If not, why treat it differently than overriding a role method, other than one's comparative level of experience and familiarity with inheritance vs. with roles?
Re:"when an inherited method is being overridden"
Ovid on 2009-10-14T14:20:50
Do you also want using 'sub foo {...}' to override a superclass method to be a compile-time failure or warning?
Many language designers are now realizing that they need to be explicit in their code behavior. For example, here's a Java method which should override a parent class method.
@Override
public int someObscureMethod() {
return 1;
}What if the parent class method takes an argument? You've not overridden the method and you may very well get an extremely hard to debug error. Now you can find out at compile time that this method isn't really overriding anything. Here's the method in C#
public override int someObscureMethod() {
return 1;
}Again, you'll find out at compile time if you didn't really override anything. You can also find out at compile time if a method overrides something but you didn't list it as overriding anything. This is very useful for the programmer. Now consider a common pitfall in C++:
#include <iostream>
using namespace std;
class Base {
public:
void doit() {
cout <<"base class"<<endl;
}
};
class Derived : public Base {
public:
void doit()
{
cout<<"derived class"<<endl;
}
};
int main() {
Base *a, *b;
a = new Base();
a->doit();
b = new Derived();
b->doit();
}That will print out "base class" twice, but that's not what you wanted. You need to declare Base::doit as virtual to get a dynamic binding instead of a static one. For a C++ programmer, when working in a derived class, the lack of annotation on the method can be very confusing and is apparently a common source of bugs, but at least by having the "virtual" keyword on the parent class you can see that it can be overridden (but it really belongs in the derived class).
You can also take a look at Eiffel for even more extreme instances where they strive to make things clear for the programmer.
In short, just because Perl makes it easy to screw up (did reciept() really override anything?) doesn't mean that it's a good thing. Many other languages are realizing this
:) Re:"when an inherited method is being overridden"
rjbs on 2009-10-14T15:02:02
I think hdp was asking an honest, simple question. I think your answer is "yes, subclassed override should also warn if not marked," with added explanation. Is that correct?
I have no strong feelings either way, although I'd rather all my existing code didn't go crazy with warnings. I just wonder what your opinion is on other forms of method occlusion or augmentation.
Re:"when an inherited method is being overridden"
Ovid on 2009-10-14T15:16:53
Yes, that was my intended answer. Many languages are moving to this, though I can see others resisting this idea.
As for "other forms of method occlusion or augmentation", I expect you're referring to things like the "before", "after", "around", "augment" and "inner" features from Moose. "augment" and "inner" must go hand-in-hand, so that's immediately visible to the programmer no matter which of the augment/inner methods they see.
The others are silent "action at a distance" but they have the advantage that if you're attempting to modify a method which isn't there, you fail almost immediately. That being said, the receiving method has no indication that it's being modified (same problem with AOP) and that's a bit disturbing, but at least you can see the roles you've composed in. This is one case, though, where tool support in the IDE would be extremely useful.
I've wondered what it would take to mark a project and load it safely (ha!) in memory to allow the IDE (Padre?) to capture this information.
Specificaly it's here: http://git.shadowcat.co.uk/gitweb/gitweb.cgi?p=gitmo/Perl-Critic-Dynamic-Moose.
I'm not sure why it wasn't released, I know that Sartak was looking for feedback. My guess is that nobody provided the feedback so it didn't get used.
I find the comments that Smalltalk doesn't have this problem because it has tools, and that you don't use Perl::Critic (a tool) so the solution that a member of the Moose Team has provided is useless, rather insulting to one of my friends a co-developers. Sartak went through the effort to create the tools the last time this discussion came up, complaining that they're not the tools you use seems petty and makes me for one less inclined to want to help develop tools like this in the future. You are a well known and well respected Open Source developer, you should know better than that.
Re:Perl::Critic::Dynamic::Moose is in git
Ovid on 2009-10-14T14:43:58
I'm very sorry I gave offense. I didn't mean to, but it's clear I phrased that very badly. I've just not been in the habit of using Perl::Critic. If given tools are ubiquitously adopted (e.g., the Smalltalk browser) or varying tools offer the same basic functionality (various Java IDEs) and are ubiquitously adopted, then tool support for features like might be very reasonable.
I suspect (though I can't prove) that many Moose developers aren't using Perl::Critic and thus aren't likely to benefit from this, thus, I agree with the traits researchers that the safety features should be in the language (or library) level rather than pushed into external tools.
Re:Perl::Critic::Dynamic::Moose is in git
perly on 2009-10-16T10:32:41
I've just not been in the habit of using Perl::Critic. If given tools are ubiquitously adopted (e.g., the Smalltalk browser) or varying tools offer the same basic functionality (various Java IDEs) and are ubiquitously adopted, then tool support for features like might be very reasonable.
Perl::Critic is an invaluable help and is well integrated for example in EPIC or Padre (with Padre::Plugin::PerlCritic). If you have pointers on how to do it with Vim or Emacs, I would be happy to ear it
:) An other possibility could be to use a precheck hooks in cvs/svn/git/whatever running perlcritic with the according .perlcriticrc (pointers to hook scripts also welcome :) As a senior developer you should definitively use it and more than that you should promote it in your team! One of the job of an senior developer is IMHO to define or help defining good coding standard and defining a perltidy.rc and .perlcriticrc is a good starting point to reach that goal. I suspect (though I can't prove) that many Moose developers aren't using Perl::Critic and thus aren't likely to benefit from this, thus, I agree with the traits researchers that the safety features should be in the language (or library) level rather than pushed into external tools.
For the record, during the YAPC::EU dinner in Lisabon, I spoke to Gabor and told him that implementing something like the Smalltalk browser in Padre would be a great thing (at least the possibility to switch between flattened and unflattened view). Please note that as a complete newbie, I am not voluntering because I consider that I have not the adequate skills
:) Re:Perl::Critic::Dynamic::Moose is in git
jplindstrom on 2009-10-18T21:22:46
Here's one way in Emacs.
Yes, it is really important that a programmer can see clearly when a trait method is being overridden - just as it is important that it is clear when an inherited method is being overridden.
I agree with Dr. Black on this point, but this is just language theory and not implementation details. It can be argued that reading the source provides this feature. Dr. Black goes on to say
Similarly, I agree that if the normal Perl environment does not provide good programming tools, then the implementation of traits for Perl ought to have done something to make it clear that a class method was overriding a trait method.
Perl::Critic is a tool in the modern Perl environment that provides this, so therefore we don't have to resort to something more drastic. You however seem to reject that in this statement
I suspect (though I can't prove) that many Moose developers aren't using Perl::Critic and thus aren't likely to benefit from this, thus, I agree with the traits researchers that the safety features should be in the language (or library) level rather than pushed into external tools.
Your conclusion (obviously) leans toward your desires, but I could just have easily had said "if they are not using Perl::Critic, they should be". If people are not willing to use the tools provided them (for free) then why should the core Moose team go out of their way to do still more work simply because someone chooses not to use another perfectly good tool?
In the end, and I have said this before, I think you are overusing roles.
While roles are an excellent replacement for multiple inheritance they possess many of the same (or similar) dead ends and pitfalls if used too heavily. It is my opinion is that using the alias/excludes feature is a code smell and that more than 3 roles being composed into a single class is a mild code smell that if not watched careful will turn into a full blown stink.
These are my opinions in the end so you can take them or leave them, but they are founded on many years of using roles in several big work projects, many CPAN modules and countless discussions had with others on IRC. I have probed the theory, but I have lived the practice.
I don't know that this will necessarily cause movement on anyone's part, but it's nice to know I wasn't totally insane in my reasoning
I think at this point, the movement needs to be yours.
You will note here that any new features for Moose must first be vetted through a MooseX module. At this point (thanks for rafl and others in the Moose team) role composition is now very hookable and you should find it much easier to create a MooseX module to so this now.
- Stevan
Re:Here we go again ...
Ovid on 2009-10-14T16:31:48
It is my opinion is that using the alias/excludes feature is a code smell and that more than 3 roles being composed into a single class is a mild code smell that if not watched careful will turn into a full blown stink.
We have no aliases in any of our role compositions and excludes are used very rarely. I also agree that they're a code smell. As for more than 3 roles, most of our classes use 4 or fewer roles. And we still do use inheritance. Though I tend to think that inheritance is dangerous, I've not gone so far as to completely excise it (except in a spike which was very successful).
One might think we're overusing roles, but so far, it's worked out extremely well with very few downsides, though I readily admit that you have more experience with them.
Re:Here we go again ...
Theory on 2009-10-14T17:12:34
It seems to me that it would be useful for Perl(5|6) to make method overriding explicit as Java has done. Similarly, I'd like to see it made explicit when you're overriding a role method with a method defined in a class. As a compile-time error.
So I think it'd be useful for you, Ovid, to implement a MooseX module that does this a al Stevan's suggestion. Then there'd be a feature that you could point people to using.
Stevan, FWIW, in Smalltalk the browser isn't just a tool, it's effectively part of the language. And although most Java programmers use tools (Eclipse), the
@Overridesyntax has still been added to the language itself. So when the tools are not part of the language (as Perl::Critic is not required to write Perl and Eclipse is not required to write Java), I think that a feature of the language should be provided for you to be explicit about when you're overriding a method.If Ovid can demonstrate this as useful in a MooseX module, so much the better.
My $0.02.
--Theory
Re:Here we go again ...
chromatic on 2009-10-14T17:21:03
As a compile-time error.
... as an optional compile-time error or warning or whatever.
Warnings and errors are affordances. A default error or warning on this behavior pessimizes legitimate, intended, and explicitly designed uses of the feature.
Re:Here we go again ...
Ovid on 2009-10-14T17:34:55
I do have MooseX::Role::Strict on the CPAN. It's already uncovered several issues here at work (we had managed to work most of them out the hard way before we switched over to this). Maybe I should add MooseX::Role::Warnings to that distribution?
Re:Here we go again ...
perigrin on 2009-10-14T18:17:20
Who's going to start championing the patch to P5P that we need this feature in the Language?
One of Moose's driving goals (which we do lose site of occasionally) is that it is Perlish. This feature (right or wrong) is not how Perl currently interprets overriding inherited methods. There are several places where Moose tries to be more strict about things than default Perl (strict/warnings enabled by default for example, though 5.11 enables strict as part of "use 5.11") and several places where things that Stevan and most of the Moose Cabal would *want* to be more strict are specifically not because it's not the Perlish way (having new() die when called as an instance method for example).
This is one of those places where I would heavily suggest that we need to stick to where being consistent with the language is important. If Perl were to start warning about overriding methods in subclasses without an explicit notation, then I would *certainly* agree that Moose could and should start doing so for Roles. Because it doesn't I think it's improper for (default) Moose to impose that burden upon developers who use Moose.
Re:Here we go again ...
chromatic on 2009-10-14T18:47:22
Who's going to start championing the patch to P5P that we need this feature in the Language?
I'm not sure it's possible with the language as currently implemented, unless for every method dispatch you walk the C3 ordering of ancestors looking for homophonic functions and accept that
AUTOLOADwill provide a lot of ambiguity or expect everyone to overrideUNIVERSAL::cancorrectly.Re:Here we go again ...
perigrin on 2009-10-14T19:49:33
Not to mention how much existing code this will break and/or start warning for.
Re:Here we go again ...
Stevan on 2009-10-14T17:40:31
One might think we're overusing roles, but so far, it's worked out extremely well with very few downsides
Well this is good to hear, perhaps all your examples of edge cases and oddities have skewed my perception of your code base. Although, your anti-inheritance stance does make me think perhaps you've gotten a little too drunk on the role kool-aid.
I guess my worry is that your not using all the tools at your disposal and therefore wanting to shape the tools you are using in such ways as to compensate. Inheritance is no more evil then roles when used properly, and delegation is sometimes a far superior solution then either of them. Hell, even multiple inheritance can still come in handy sometimes.
And while I agree that strictness of composition is useful, there is a tradition in the Perl community of giving you enough rope and Moose tries to strike a balance between enough rope and saving you from yourself. But the fact most of your issues stem from (what I perceive as) more extreme or uncommon usage patterns, it makes me think a MooseX module or optional "strict" setting is more appropriate.
- Stevan
Re:Here we go again ...
Ovid on 2009-10-14T16:44:54
Oh, and as for "Here we go again", I'm not asking for or expecting any changes to Moose in this regard. I only raise this issue because there seemed to be some confusion on the part of some (perhaps myself) about why the behavior is the way that it is.
Re:Here we go again ...
Stevan on 2009-10-14T17:07:52
Yeah sorry, that was an bad title written off the cuff and then not really considered, I didn't mean it as snarky as it sounds:) Re:Here we go again ...
Ovid on 2009-10-14T17:27:21
No worries. I do that all the time myself -- to my constant chagrin
:)
