In Pondering Role Organization, I was trying to figure out if I want an abstract base class or not.
He's code with the abstract base class:
package Order;
use Moose;
extends 'My::ResultSource';
with qw(
DoesAuditing
DoesCustomerSearch
);
...
Here's this code using only roles:
package Order;
use Moose;
with qw(
DoesResultSource
DoesAuditing
DoesCustomerSearch
);
...
We've elected to go with an abstract base class here at the BBC. I now realize this is a mistake. In fact, I think it's a bad mistake, the sort which an amateur chess player might make.
So let's saying that you're playing chess and you open up with a classic "Ruy Lopez". It's hundreds of years old, it's a strong opening, and it's very powerful. After about 6 or so moves, you decide you don't like it and you switch to a hyper-modern opening. In the Ruy Lopez, you attempt to control the center by occupying it. In hypermodern strategies, you control the center from the periphery. Each has strengths and weaknesses and it's not impossible to switch from one to another within in a game, but if you do this all the time, you're probably losing more often then you win. This is because of an age-old rule: given two players of equal skill, the player with a bad plan will beat the player with no plan. Arbitrarily switching strategies in the middle of a game is equivalent to having no plan. Mixing inheritance and roles is like trying to figure out which strategy you want to adopt.
The only benefit I could see to using an abstract base class is to be able to say $object->isa('My::ResultSource'), but what benefit does this have over $object->DOES('My::ResultSource')? Absolutely none that I can see. In fact, by using an abstract base class, we lose the ability to get composition time detection of method conflicts. Silently overriding methods is a mistake.
<digression>
Not everyone agrees that silently overriding methods is a mistake. Here's my reasoning, based upon many hours of painful debugging.
Many languages such as Java, C#, Ruby and others don't allow multiple inheritance (MI). This is due to how easy it is to silently "skip" methods. Every language which forbids MI provides some way of working around this, though, because some problems cannot be solved merely through class composition. A later version of my Refactoring With Roles talk (not online, I'm afraid), gives one such example.
Many languages, even those which forbid MI, also force you to be explicit about whether or not you're overriding a method. C++ fails because you must declare a method as "virtual" in the base class, so the concrete class has no visual indication of this (outside of an IDE helping out), so the programmer has to root around in the base class. Even compiling the code won't necessarily help because you may have introduced a bug that even your tests won't necessarily have found.
Eiffel actually does a great job of managing much of this, forcing the programmer to be explicit about everything, but they still lock you into a class hierarchy and the attendant composition problems which may arise. As biologists have discovered in trying to classify animals into species, genus, family, etc., a simple hierarchy doesn't work and OO graphs don't necessarily map well to the "genes" of an object. Many, many languages struggle with this problem and get it wrong, or at least have varying degrees of "right". It's a hard problem and one that needs to be solved at the language level, not the developer level.
As a result, there's a certain minimum level of complexity which any large-scale system and when the computer can find potential problems -- particularly when it's close to compile time -- this is far better than requiring the programmer to always look for those potential problems. Remember, we have computers for a reason. They do our grunt work for us. And here's the key to all of this: because the programmer, under a tight deadline, with complex code, often poorly documented, overrides a method they missed, despite diligently reading as much as they can, silently overriding said method hurts him, particularly when the fix is both trivial and well-understood. With roles, the fix is trivial and well-understood.
</digression>
To me, the silent overriding of methods is the final nail in the inheritance coffin for Perl. However, I realized after a while that there are other considerations, some unique to Perl and some not.
In Perl, we don't distinguish between class and package names. If a package name is brought in via use or require, we are tightly coupling this package name to a path on disk. As a result, we are generally coupling OO behaviors to disk layout. This is a marriage made in Wonderland. It's not necessarily wrong, but it's decidedly odd. Frankly, I want my disk layout (different directories) to reflect the different facets of the system I'm working on, not necessarily the class layout.
By dispensing entirely with inheritance and relying solely on roles and forbidding silently overriding methods, we can guarantee a certain minimum level of sanity at composition time. This, interestingly, is one of the strongest benefits of static typing. By properly modelling behaviors and forbidding ambiguity, we can mitigate one of the most reasonable objections to dynamic languages.
We also no longer have hierarchies. Objects can be placed anywhere in your directory tree to properly reflect business requirements rather than technical ones. Yes, we'll still have coupling across directories/packages/classes, but you always have that in systems: the bits have to be able to talk to one another. That being said, class composition errors go away. MRO problems are gone. Open up a class and you see all of its behaviors listed at the top[1] rather than rooting around through base classes, trying to remember their order and hoping you know what overrides what. Complexity management for large-scale systems becomes easier.
I realize that much of this seems like "pie in the sky" talk, but so far, it's working out well for us. This is real code which is solving a real problem. No, this is not a Holy Grail or Silver Bullet, but it seems like an huge step in the right direction. I'm quite pleased.
--
1. This may imply that roles do not consume other roles. This is the strategy I am following. I only want a role to consume another role if that other role is merely an identifier and perhaps an interface:
Package DoesSearch::Tags; use Moose::Role; with 'DoesSearch'; # no behavior. Just a name.
[When] the computer can find potential problems -- particularly when it's close to compile time -- this is far better than requiring the programmer to always look for those potential problems.
This is not a case where the computer can identify problems with certainty. The compiler cannot judge your intent. Did you make a typo in the name of a class-local method such that it collides with the name of a composed method? Did you forget to read the documentation? Did you do it deliberately? Did someone upgrade a role and not tell you?
The compiler has no access to this information and can only guess.
That's not a strategy for robust programming, and it's a terrible strategy for user interfaces. (Imagine if Perl warned every time you slurp a file into an array. Sometimes that's a mistake -- perhaps often, it's a mistake. It's not always a mistake.)
The compiler can only know that it should warn on every class-local method declaration excluding composed methods if you tell it that it should warn. That's easy to do; enable an optional warning.
Re:On Warnings
sartak on 2009-04-24T20:12:45
Imagine if Perl warned every time you slurp a file into an array. Sometimes that's a mistake -- perhaps often, it's a mistake. It's not always a mistake.
Imagine if Perl warned you every time you used a package-scoped variable only once. Sometimes that's a mistake -- perhaps often, it's a mistake. It's not always a mistake.
Imagine if Perl warned you every time you redefined a function (for example to memoize or inline it). Sometimes that's a mistake -- perhaps often, it's a mistake. It's not always a mistake.
Imagine if Perl warned you every time you recursively called a function more than 100 times.
Every time you used comma in a qw//.
Every time you leave an eval with next.
Every time you use @{ } or %{ } on a variable that happens to share a name with a builtin.
Every time you include some code directly after an exec call, so you can throw a proper error if the exec fails.
It's a wonder why anyone ever turns on these bothersome warnings. Some modules even have the gall to enable them for me without my explicit permission.
enable an optional warning
I encourage you to read this insightful blog post about why optional warnings are short-sighted. It is every bit as accurate when its subject is swapped for Moose.
* * *
Please don't take this the wrong way, chromatic. I respect you, your writing, and your contributions to free software. I just tire of this outrage over a warning whose purpose is to help users write more maintainable and clear code. "No surprises" is a laudable goal not only during release, but also between releases.
Re:On Warnings
chromatic on 2009-04-24T21:14:33
I encourage you to read this insightful blog post about why optional warnings are short-sighted.
I call: The Value of a Warning.
If you believe that the only purpose of roles is compile-time, warning-safe mixins, you're missing at least half of their power.
The purpose of roles -- the real, get your hands dirty, oh wow this is an epiphany moment of roles -- is to produce a type system that tracks the capability of entities without dictating their particular implementation of those capabilities.
In a true role-based system, your entities perform various roles by inheriting from other classes which perform those roles, by composing in the behavior of various roles, by delegating specific activity to other entities which perform those roles, or by reimplementing part or all of the behavior encapsulated in the role. Obviously in the latter two cases you must explicitly declare that your entities perform those roles, but you do not compose in the behavior of those roles, at least in part.
This has been an explicit design goal of the system from the very start.
Roles should not and cannot dictate specific implementation details of entities which perform the roles, otherwise they're merely somewhat smarter mixins. It's nice to have somewhat smarter mixins -- they offer advantages -- but that has never been the sole point and goal of roles.
The moment you throw a mandatory default warning on two of the four appropriate and specified uses of roles, you've penalized them. You're subtly encouraging people not to use the most important and most powerful features of roles! You're actively discouraging people from taking advantage of allomorphism using well-established and long-recommended design techniques explicitly made safer and more understandable by roles.
It's difficult enough to convince people not to cram everything into a complex hierarchy without you strongly hinting that delegation and allomorphism are dangerous or complicated or prone to abuse.
If that's your intention, by all means proceed -- but please be clear about it and don't justify your decision by pretending that the compiler can magically guess their intent, because we all know if it were true, we wouldn't even need roles.
Re:On Warnings
sartak on 2009-04-25T13:24:32
You're actively discouraging people from taking advantage of allomorphism using well-established and long-recommended design techniques explicitly made safer and more understandable by roles.
I responded at my blog with a description of both sides of the argument. The gist of it is: the warning is gone, and we will support the warning Ovid needs (and much more) as Perl::Critic policies.
Re:On Warnings
Ovid on 2009-04-24T23:02:34
Please note that I'm not trying to change your opinion here because it's obvious that we strongly disagree here. I provide this for anyone else who may be reading this.
This is not a case where the computer can identify problems with certainty. The compiler cannot judge your intent.
And that's why the warning is so desperately needed. If I inherit from A and B and both provide a "foo" method, I usually get the one that I've inherited from first. One could argue that I forgot to read the documentation or that I did it deliberately, but just like the composition problem I list above, there's no way that the compiler can know this, so it should warn, but with MI, it doesn't and things silently break. We have a way out of this trap with roles.
Did you make a typo in the name of a class-local method such that it collides with the name of a composed method?
If I write a method that accidentally collides with another method, it implies that I'm adding behavior and not trying to override behavior. Thus, a warning is warranted because what I'm doing doesn't match what I want to do.
Did you forget to read the documentation?
I'm particularly keen on focusing on real-world usage here. Documentation is often incomplete. Developers often don't read documentation. Developers (especially me!) often miss crucial bits of documentation. Sometimes I read the documentation several months ago and have heavily used the code and am not aware that it's changed. We are approaching 600 packages and no, I don't remember all of them and having to reread their docs every single time I work with them is not an option. I have to get stuff done. Having a feature that hopes that even conscientious developers haven't made a mistake is wrong.
Did you do it deliberately?
Did I do what deliberately? If you mean "override the method", then this implies I knew the method was there and explicitly excluding it takes about two seconds and let's other developers come along and instantly see that there's an excluded method. This is a benefit because it increases the readability of the code.
Did someone upgrade a role and not tell you?
The I absolutely want that warning! That means that someone else is trying to add behavior and that behavior is presumably desired instead of silently ignored! This is probably the strongest argument in favor of a warning.
The compiler has no way of divining my intent, so the compiler should say "hey, I'm not sure what you meant here, would you please be explicit?"
And your "slurp a file into an array" argument is completely different. You're not losing information there. You are losing information when you silently override a method. Please provide an example where we silently discard information for a better comparison. So many languages struggle with class composition issues and get it wrong repeatedly. I don't want to see that happen with roles.
I've worked with you before and I know you're a good enough programmer that it's quite possible that many of these issues don't apply to you. For me, I work with poorly documented code. I forget about methods in documentation I've read before. I update code and run my test suite and don't notice that there's a corner case I've missed. My code's not perfect and I want the language to help me out when it sees something that it knows is dodgy and is trivial to fix. The added benefit is that every programmer subsequently maintaining that code can explicitly see what's going on. That's a joy we often don't see in dynamic languages.
Re:On Warnings
chromatic on 2009-04-24T23:30:42
If you mean "override the method", then this implies I knew the method was there and explicitly excluding it takes about two seconds and let's other developers come along and instantly see that there's an excluded method. This is a benefit because it increases the readability of the code.
Yet when I override multiple methods (or all of the methods) from a role for the purpose of complete allomorphism or delegation, your approach means that I have to exclude every one of them explicitly, which is mere busy work and syntactic noise and actively impairs the readability of the code, disable a warning against a valid design goal of roles, or I live with a compiler warning which tells me that I have very well done precisely what I intended to do.
Tell me with a straight face that it's not crazy to apply a role to a class and then immediately list all of the methods you don't want to compose from the role into the class because they appear in the class immediately after the role application.
As I said in my other reply, if your intent is to subtly suggest that people treat roles as mere super mixins, the on-by-default warning is a great idea.
The compiler has no way of divining my intent....
Yes. That's the problem exactly. If you know that you will never take advantage of explicit class-local overrides and that you know that any class-local overrides are mistakes, by all means enable the warning locally in your own code. It's no problem there. You have a good heuristic for knowing that collisions are error conditions, because your coding standards (I presume) recommend against class-local overrides.
You don't know that about my own code, so please don't make the compiler try to help me by warning me that my code behaves exactly as I intend it to behave -- and not just the code itself, but the intent and design of roles.
Re:On Warnings
Ovid on 2009-04-25T07:51:11
For the case of overriding everything or almost everything and the role is not simply an interface, then yes, the work to exclude all of the role's methods would be annoying. That's the only interesting argument I've heard from this entire discussion and had the discussion started out with this, then things might have gone easier.
The problem is that your solution is still throwing away information, my solution can be cumbersome at times. So the reality is that this is a syntax issue. If a good, clean syntax were found which makes overriding a role's methods explicit and non-cumbersome but still warns about ambiguity, then both concerns can be addressed. I don't (yet) know what that syntax would be, but your solution of ignoring the real, existing, many hours of debugging problem that we've had at the BBC trying to find those silently overridden methods doesn't work for me.
Re:On Warnings
chromatic on 2009-04-25T20:04:35
If a good, clean syntax were found which makes overriding a role's methods explicit and non-cumbersome but still warns about ambiguity, then both concerns can be addressed.
mst has some very nice examples using
MooseX::Declare, where you provide a block after applying a role. Any method you define in that block very obviously takes precedence over methods composed from the role.
... your solution of ignoring the real, existing, many hours of debugging problem that we've had at the BBC trying to find those silently overridden methods doesn't work for me. I'm not ignoring it. I've consistently said that an on-by-default warning is the wrong approach. An optional warning is just fine. Make one, use one, enforce it in your coding standards -- great! You should have that option. Sometimes I may use that option too.
You know how you use roles. My concern is how everyone uses roles and what the design of Moose roles promotes and discourages.
Re:On Warnings
zby on 2009-04-25T09:14:33
Hmm maybe we need some examples here. I am just starting to use Moose, so I must be missing something, but I cannot imagine someone wanting to override all methods of a used role.Re:On Warnings
Ovid on 2009-04-25T09:47:14
I can easily imagine this. If you simply want to say "I provide this behavior" but your implementation differs significantly, you might override all of the roles (this is different from an interface because the role can also provide an implementation). The issue is that this is the use case that chromatic wants to support and I'm against silently discarding behavior.
Re:On Warnings
zby on 2009-04-25T11:45:23
I don't understandy why to declare "I provide this behaviour" someone would use a Role. Maybe it comes with more practice - that is why I ask for concrete examples.Re:On Warnings
Ovid on 2009-04-25T12:42:36
Let's say that you can serialize an object as HTML. You might use the role Role::Serializable::HTML with a &serialize method. Then someone can ask:
if ( $object->DOES('Role::Serializable::HTML') ) {
print $object->serialize;
}By providing a name for the behavior, you are guaranteeing to someone that you provide a &serialize method and that it will serialize the object as HTML.
However, and this is chromatic's concern, it's quite reasonable that the object might want to use that role to provide a named behavior, but it might need to provide its own &serialize method because the role's &serialize method might not be sufficient for that object's needs. Roles provide named behaviors that make introspection easier.
Re:On Warnings
chromatic on 2009-04-25T20:07:54
I've written a lot of mock objects, and I've had to work around a lot of code which performs its own type checking. That can interfere with the work of mock objects. I plan to write an article with more concrete examples soon, but for now I hope it's not too abstract to say that the question "Is this entity a member of an inheritance hierarchy at this point or lower?" is much less interesting and much more difficult than the question "Does this entity perform the behavior I expect?"
Re:On Warnings
btilly on 2009-04-25T11:20:11
Tell me with a straight face that it's not crazy to apply a role to a class and then immediately list all of the methods you don't want to compose from the role into the class because they appear in the class immediately after the role application.
Tell me with a straight face that it's not crazy to use a module and then be forced to list all of the methods that you want to import because it didn't define @EXPORT. Particularly when the only reason why someone would want to use that module is to get access to those functions.
In case the parallel isn't obvious, the common thing here is action at a distance. If you have a number of roles with complex behavior, it isn't obvious what behavior any given role will have. Which means that it is (as Ovid discovered) very easy to have accidental conflicts that Do The Wrong Thing. It doesn't take much typing to make accidents impossible, so to me the tradeoff between whether it is better to be convenient or to require a declaration is pretty clear.
Admittedly not everyone will agree. Not even every decent programmer. I claim that is because the convenience is obvious, and the potential for repeated bugs from action at a distance is not. Again I raise the similar issue of how many programmers have thought that @EXPORT is a good thing. Somewhat more controversially, I would also point to how many programmers have made the mistake of overusing inheritance. In both cases the perceived convenience is usually outweighed in the long run by the potential for bugs from action at a distance.
Just because something is convenient, and someone else thought that you should take advantage of that convenience, doesn't mean that taking advantage of the immediate convenience is a good idea.
Re:On Warnings
chromatic on 2009-04-25T20:13:41
Tell me with a straight face that it's not crazy to use a module and then be forced to list all of the methods that you want to import because it didn't define
@EXPORT. Particularly when the only reason why someone would want to use that module is to get access to those functions.The important difference is that roles have always had the explicit design goal of not enforcing any particular implementation decision to take advantage of them.
Re:On Warnings
btilly on 2009-04-25T20:48:22
The fact that someone has a legitimate design goal that creates action at a distance doesn't necessarily make the decision to create action at a distance a good idea.
In a similar spirit I would not be opposed to an optional warning if my subclass overrode a parent class's method and didn't, say, provide an attribute that says, "Yes, this is an intentional override, don't warn." You might find having to type
:override annoying, but I would find it invaluable. (I also know that I have a snowball's chance in hell of getting it to happen...) Re:On Warnings
chromatic on 2009-04-25T22:59:10
I believe it's easier to argue that inheriting or composing in methods is more action at a distance than declaring them locally. Perhaps prototype OO systems are clearest by this metric.
I noticed you are using a role called 'ResultSource'
Are you using DBIx::Class?
If so, how are you integrating it with Moose? Are you extending the DBIx::Class'es or are you wrapping the storage object (the row)?
That is, do you have two classes for each business entity: one for the model and one for the storage (DBIx::Class)?