Day 247: Final day at Liz's - Coroutines, siglists, rebindable bindings

autrijus on 2005-11-06T21:28:15

Today is the last day of my stay at Liz and Wendy's. It was very enjoyable and productive, and I'm very grateful to their warm hospitality.

Liz and I had a SubEthaEdit session on coroutines today, with the brief notes merged to S17 (Liz is working on Englishify them at this moment).

Basically, a coroutine is an object that supports a .start method that initiates the lightweight thread behind it, and return a object that may be manipulated like any other threads. If you call it like a nullary function, it activates the thread until it hits yield, in which case it gives the control back to the caller. The return inside the thread will end itself immediately. Either way, if you attempt to call a finished thread, an exception will be thrown.

Then we get the extra magic of calling a Coroutine objects like a Code object. This will trigger &Coroutine::postcircumfix<( )>, which will take care of calling .start again after the thread has finished, and arrange it so the next call to the coroutine will resume the ongoing thread instead of starting a new one.

Finally, there is the question of calling an ongoing coroutine with parameters. I think rebinding the parameters makes sense (as you can inhibit it using is copy or explicit { my $x := $OUTER::x }), but iblech's idea of having yield returning the Arglist makes sense too. In any case this is userland code, so one can change the behaviour with a trait or a subclass of Coroutines.

On the PIL2 front, I looked at the three special forms (Assignment, Binding, Apply) and decided to unify them into method calls, so the user can override them all inside the same object model -- "Everything is an object."

The Apply form is easiest, as it's just method postcircumfix<( )> that takes an Arglist and gives back an Arglist. See the new S03 for more about the cool things you can do with Arglists.

Assignment is also manageable under the container model. It will be method infix:<=> on Array, Hash, Scalar objects. The List objects constructed with infix:<,> will also respond to this method, to implement the ($a, $b, $c) = (1, 2, 3) forms.

Binding almost feel like it should be a non-user-overridable special form, except we remembered the permanently-semi-official idea of Siglist objects, constructed with the :() notation. It turns out that if we specify binding as a method on the Siglist object, and that the compiler desugars LHS := RHS always as :(LHS).infix:<:=>(RHS), it will all Just Work. Under this regime, 3 := 4 will still be a compile-time error, as :(3) would make no sense -- the macro that turns sub (3) {...} into sub ($ where {$_ ~~ 3}) {...} would not apply to bare Siglist construction forms.

So, PIL2 is looking good with fewer nodes and tighter object integration now. Maybe we are not yet at io's zero keywords level, but it's getting dangerously close. Note that all of this is outside specced domain -- we'll need to work them back to Synopses and tests, and run them through p6l and @Larry.

Of course, to recover a respectible performance under this regime, we need to rely on optimizers, early binding, and/or type inference. But that's two milestones ahead, so let's get the dynamism part right first...


yield result

jmm on 2005-11-07T14:23:46

Passing the new arguments in as the return value of the yield is the only way that makes sense to me. Rebinding the original arguments might be very far removed from the yield statement (the coroutine might have started in one block of code, but the yield might be the n'th sub child, where the original arguments are totally inaccessible and will not be accessible until all of the nested subs have finished yielding partial results and have returned back to the the grandparent code that actually has access to those arguments.

Re:yield result

jmm on 2005-11-07T14:33:30

Additionally, it makes sense that the argument list for "start a new sequence" is different from "continue to find the next sequence element". The first argument list will contain arguments that are global to the entire sequence; while the second will be more local in meaning. The canonical coroutine example is a walker for a binary tree. The call that starts walking the tree might have an argument that chooses pre-order, in-order, or post-order traversal. That argument would not make sense for one of the get-next calls, but other arguments might (perhaps a temporary filter argument, for example).

Re:yield result

autrijus on 2005-11-07T15:55:37

Hmm, maybe the ongoing subroutine needs a &coro.next() method that simply resumes the yield, and then calling &coro(args) will pass args as yield's return value, just as you and iblech had argued for. Rebinding can then take place if the user asks for it, with an is rebound trait. What do you think of this?

Re:yield result

jmm on 2005-11-07T17:40:21

Having both a functional and object interface to a coroutine is a separate issue.

In the functional interface, if you don't have anything to pass as the return value from the yield, just pass nothing. So, &coro(args) will cause the yield to return (args) and &coro() has it return nothing. No need for a method to distinguish. Generally, I think that whenever you look to rebind the initialization arguments, you'd generally prefer to start a separate coroutine instance with different initialization args; and then resume either of the two coroutine instances as appropriate.

I'd tend to prefer that a coroutine have two signatures - one for instantiating a new instance of the coroutine, the other for resume an ongoing instance. (In that regard, perl 5 with its typical lack of signatures works well.)

For an object interface, these two signatures could be provided as two methods - called something like (start/init/new) and (resume/next/continue).

There is also the tricky interface of creating a new instance the first time the coro is called, and then resuming it on subsequent calls, with perhaps an exception if it is called again after it terminates before creating another new instance. This interface makes for a nice sugar for simple uses and should be built on top of the more general interface, so that it doesn't get in the way for more complicated usage.