All the Perl that's Practical to Extract and Report

Aspect 0.90 - Aspect-Oriented Programming for Perl, rebooted

Alias on 2010-05-27T02:44:01

After two years of development at $work, we've finally completed our program of stability work. To the great credit of our 7 man Perl team, we've managed to reduce the (predicted and actual) downtime to about 1 hour per year for our billion dollar ecommerce site that you've never heard of.

With stability solved, our focus now turns to usability. And unlike downtime, which is easily measurable, we don't really have any decent metrics for usability. So in preparation for this massive new metrics development push, for about a year now I've been hacking away on Aspect.pm, Perl's Aspect-Oriented Programming framework.

My plan is to use this to write performance and metrics plugins for a generic Telemetry system, which would less us turn on and off metrics capturing for a library of different events, driven by configuration on the production hosts.

If you have never encountered Aspect-Oriented Programming before, you can think of it as a kind of Hook::LexWrap on steroids. The core "weaving" engine controls the selection and subjugation of functions, and the generation of the hooking code, and then two separate sugar-enhanced mechanisms are provided on top of this core to define what functions to hijack, and what to do when something we hit one of them.

The main power of AOP is in the ability to build up rich and complex hijacking conditions, and then write replacement code in a way that doesn't have to care about what is being hijacked and when.

This ability to manipulate large numbers of functions at once makes it a great tool for implementing "cross-cutting concerns" such as logging, tracing, performance metrics, and other kinds of functions that would normally result in having a bucketload of single lines scattered all through your implementation.

It also lets you do semi-evil "monkey-patching" tricks in a highly controlled manner, and potentially limited to only one single execution of one single process, without having to touch the target modules all the time everywhere.

For example, the following code shortcuts to false a particular named function only if it isn't being called recursively, and randomly for 1% of all calls, simulating an intermittent failure in the interface to some kind of known-unreliable hardware interface like a GPS unit. 

use Aspect;

before {
    $_->return_value(undef);
} call 'Target::function' & highest & if_true { rand() < 0.01 };
While the original implementation of Aspect.pm wasn't too bad and did work properly, it had two major flaws that I've been working to address.

Firstly, it was trying to align itself too closely to the feature set of AspectJ. Unlike AspectJ, which does aspect weaving (installation) at compile time directly into the JVM opcodes, Perl's Aspect.pm does weaving at run-time using function-replacement technique identical to Hook::Lexwrap.

This difference means there's certain techniques that AspectJ does that we can't really ever do properly, but there are other options open to us that couldn't be implemented in Java like the use of closures.

For example, with Aspect.pm you can do run-time tricks like that rand() call shown above. Since the if_true pointcut is also a closure, you can even reach have it interact with lexical variables from when it was defined.

This makes for some very interesting options in your test code, allowing lighter alternatives to mock objects when you only want to mock a limited number of methods, because you can just hijack the native classes rather than building a whole set of mock classes and convincing your main code to make use of them. 
use Aspect;

# Hijack the first four calls to any foo_* method
my @values = qw{ one two three four };
before {
    $_->return_value(shift @values);
} call qr/^Target::foo_\w+$/ & if_true { scalar @values };


The second problem I've fixed is the speed. The original implementation made heavy use of object-orientation at both weave-time and run-time.

Logically this is a completely defensible decision. However, from a practical standpoint when you are going to be running something inline during function calls, you don't really want to have to run a whole bunch of much more expensive recursive methods.

So I've restructured much of the internals to greatly improve the speed.

I've dumbed down the inheritance structure a bit to inline certain hot calls at the cost of a bit more code.

I've also implemented a currying mechanism, so that run-time checks can safely ignore conditions that will always be true due to the set of functions that ended up being hooked.

And finally, rather than do method calls on the pointcut tree directly, we now compile the pointcuts down to either a string parameter-free equivalent functions.

In the ideal case the entire pointcut tree is now expressed as a single-statement Perl expression which makes no further function calls at all.

This not only removes the very expensive cost of Perl's function calls, but having everything in a single expression also exposes the boolean logic to the lower-level opcode compiler and allows culling and shortcutting at a much lower level.

With these two major improvements in place, I've also taken the opportunity to expand out the set of available conditions to include a number of additional rules or interest to Perl people that don't have equivalents in the Java world like tests for the wantarray'ness of the call. 
use Aspect;

# Trap calls to a function we know behaves badly in list or void contexts
before {
    $_->exception("Dangerous use of function in non-scalar context");
} call 'Target::function' & ! wantscalar;


In addition to the expanded functionality, I've also massively expanded the test suite and written up full (or at least nearly full) POD for all classes in the entire Aspect distribution.

To celebrate this near-completion of the first major rewrite phase, I've promoted the version to a beta'ish 0.90.

There's still some work to do before release 1.00. In particular, at present the point context objects still work to a single unified set of method calls, regardless of the advice type.

This means that advice code can attempt to interrogate or manipulate things that are irrelevate or outright illegal in certain contexts.

Cleaning this up will elevate these bad context usages to "method not found" exceptions, and will also allow prohibiting the use of pointcut conditions in contexts where they are meaningless (such as exception "throwing" clauses for before { } advice when no such exception can exist).

It should also allow faster implementations of the point context code, because it won't need to take multiple situations into account and can specialise for each case.

Once this is completed, and some additional test scripts have been adding for particularly nasty cases, I'll be able to do the final 1.00 release.

Re: Aspect 0.90 - Aspect-Oriented Programming...

cosimo on 2010-05-27T06:57:13

Adam, this is cool.

However, I'd be interested in the nature and details of your never-heard-of billion dollar site stability work. Any chance of you talking about that?

Tiny nit

tsee on 2010-05-27T07:55:53

I should file this as a bug or fix it in SVN myself, but there's a small typo in the synopsis of 0.90: s/\$wormhole/\$aspect/

Some issues with AOP

Ovid on 2010-05-27T08:05:23

One thing I think you should note in your documentation is that it's very important to realize that a software must not rely on aspects to function. That is to say, if all pointcuts fail to match, the software must still be correct. Aspects should generally add behaviour, not alter it.

For example, one common example Java programmers use is to mark code as thread-safe. However, if a pointcut fails to match a function which must be thread-safe, the match fails silently. As a result, previously thread-safe code may no longer be thread-safe, but the programmer won't notice this and the tests are highly likely to not catch this.

So if your pointcuts match all methods named foo_\w+ and some bright programmer fails to notice this action at a distance (and who would fail to notice action at a distance?) and renames some functions that the pointcuts previously matches, the behaviour of the code is altered, but it's awfully hard to notice this.

This is one of the reasons I like roles. Consider this:

package Role::Munge;
use Moose::Role;

requires qw(some_method);

before 'another_method' => sub {
    my $self = shift;
    my $stuff = $self->some_method;
   # do more stuff
};

If some enterprising programmer renamed either 'some_method' or 'another_method', this code fails at composition time and gives composition safety. AOP silent-failure modes and action at a distance were one of the reasons roles (traits) were created in the first place.

So if you find AOP useful, that's great. Just note that if your system is incorrect without aspects, you have a major design flaw.

Re:Some issues with AOP

Alias on 2010-05-28T00:02:58

I concur.

I think while AOP shows promise for stuff beyond just tracing, logging and what not, I can't see how you could scale it without similar problems to many other purely declarative systems like XSLT where actions happen in parallel and at a distance.

I'll look at adding some kind of Aspect Caveats section to the documentation.

Bitwise operators?

jjore on 2010-06-02T03:17:02

Hey, what's with the bitwise operators? Would ordinary boolean operators work here or are you doing this for some invisible-to-me precedence reason?

Re:Bitwise operators?

Alias on 2010-06-03T00:34:14

Historical reasons.

The bitwise operators were the ones that the Aspect library used when I took it over.