All the Perl that's Practical to Extract and Report

Hackathon Days 2+3: Report

chip on 2005-06-13T21:05:05

As I suspected, things quieted down for me quite a bit over the last two days ... they've gone from insane to merely busy. While Autrijus has been releasing Pugs, Leo and I have been designing things and considering priorities.

Candy first. Leo came up with a sane call/return convention for Parrot. After some discussion with Autrijus and me, Leo has now implemented a useful working subset of the final feature. The new call/return convention involves some new opcodes, does automatic type conversion and array flattening and ... well, read my update of PDD03 for yourself. (Yes! There are docs! A PDD even! And I've updated it! OMG! PDD! WTF?! If there's a fellow Parrot user near you, check to see if he needs resussitation...)

I've also been pondering priorities, and otherwise putting thought into the least hacker-like and yet most crucial Fearless Leader task: Pondering who Parrot should serve, and how. I'm not planning to live in Cloud City -- I'm a troglodyte hacker at heart -- but making design decisions where there is no clear right answer requires a usable utility metric. Otherwise you just end up floundering around and chasing cool things.

Actually, that's one class of user: the Magpie. ("Oooh! Shiny!") I've modelled the Parrot users I could think of in the Parrot Model Users document. Check it out. Who have I left out? Who matters the most? Don't worry about getting the right answer on those questions. There probably isn't one.

And boy have I learned a lot about continuations. Autrijus has been diving deeply into the literature on this for a while, and I've been picking his brain at every opportunity. (You can pick your friends, and you can pick their brains, but you can't pick your own brain. Or something like that.) The p6i discussion on the subject has been, well, mixed, not least because I've taken a few mail exchanges to figure out exactly what we need and want.

The short version of the situation with continuations is:

  • Parrot will support full continuations, not just escape continuations. That is, continuations will not be limited to jumping up the current call stack.
  • Parrot continuations will not require any new save and restore operations to work properly. One of the advantages of a virtual machine is that its state is always already saved. Right there in, you know, memory. Invoking a continuation is pretty much just switching your "current activation record" pointer.
  • Parrot's register allocator will have to be changed to allow for the extra flows of control produced by continations. Essentially, no register assigned to a pseudo-register that is used across a function call boundary can be aliased to any other pseudo-register. This would have been a problem if not for the register expansion we already agreed to implement. Now it's no big deal.
  • Finally, most PMC operations (except function call, of course) are guaranteed not to invoke full continuations. They can do the escpae continuation thing (i.e. throw an exception), but that's all. This is a requirement for two reasons: First, if those functions invoke Parrot code, they do so in a separate run loop, which makes the full continuation implementatin a real pain; but second, because we don't want to send the register allocator totally around the bend by making every PMC operation a boundary like function calls already are. This implies that if you want to invoke a continuation, you can't do it in your TiedArray::FETCH function. I don't think we'll get a lot of complaints about that.
And that's all the news from Herrnbaumgarten. Guten Nacht.

More users

Manuzhai on 2005-06-14T12:21:31

Okay, here are more Model Users:

=head2 "Sam": Python implementor

Sam has a favorite language, Python, and he wants to target it to Parrot.

Sam:

* is only interested in Parrot if it's faster than CPython

=head2 "Rasmus": PHP implementor

Rasmus has a favorite language, PHP, and he wants to target it to Parrot.

Rasmus:

* wants to replace Zend with Parrot for PHP 6

Re:More users

milardj on 2005-06-14T14:07:12

I think you raise a great point for reminding people that Parrot is not just a Perl VM.

I'm not sure you have "Sam's" (and lets add "Matz") motivation correctly. Speed is obviously a factor but the biggest thing I keep hearing (especially from the Ruby crowd) is access to CPAN. So speed *and* transperant language interoperability where by the latter I mean if you provided stubs for the external code then your script would run as is with CPython or the Ruby interpreter.

To me language interoperability is still the great unknown in terms of mechanics and in terms of how tightly|loosely contstrained it will be.

I remember that one of Sam's constant complaints was that Perl6-centric decisions were being made that made it more onerous to accomodate Python. I have no details just dimly remembered (!sharp!) e-mail exchanges between Sam and Leo.

Re:More users

chip on 2005-06-16T13:38:36

Well, Parrot will never be Perl6-specific, but without the Perl installed base, Parrot won't ever reach its potential.

"Too many priorities is no priorities."

Re:More users

treefrog on 2005-07-22T11:58:43

Hmmm- I've been lurking around parrot for a while (being a ruby type bod). Really glad to see the directions that things are going in at the moment, and I really appreciate seing so much documentation of what Parrot is trying to do (Ok, so I am in telecoms!).

So a couple of comments:

1. Users Sam, Rasmus, Matz etc are important. A good VM model will last forever (or nearly so).

2. While I'd claim to be a User Matz type, I am also a User Tom. While I do try and keep up with the design docs, I have not been involved with development (although I have read parts of the source code). I am interested in the lightest weight sustainable subset of Parrot that I can find. I want to have a minimal Parrot, and I want to be able to keep it very well integrated to an OS kernel.

Best regards

treefrog

Ilya user type tweak

geoffrey on 2005-06-15T17:40:02

I mentioned this on #perl6, but I think you missed it -- the high-performance user ("Ilya") may be doing bandwidth-intensive stuff as well -- it's important then to have efficient access to packed data streams.

This becomes especially important when each data element requires very little computation, but there are a LOT of data elements to go through.

Hmmm, perhaps this should be a new user type . . . .

Re:Ilya user type tweak

chip on 2005-06-16T13:23:37

Lots of calculation and lots of data movement are closely related, but I don't know if they're the same or not. If you're moving so much data that the mere movement of it is a challenge from Perl, actually processing any of it substantially is probably a C-language issue (e.g. mpeg compression).

Perhaps you could contribute an "Ilya Prime" sub-profile?

Re:Ilya user type tweak

geoffrey on 2005-06-16T21:49:05

$ svk diff
=== docs/req/model_users.pod
====================================================== ============
--- docs/req/model_users.pod  (revision 13090)
+++ docs/req/model_users.pod  (local)
@@ -103,6 +103,38 @@
   a favorite feature set and stick with it

+=head2 "Ilya Prime": Intensive Memory Bandwidth User
+
+Ilya Prime writes high-performance memory bandwidth-bound code,
+typically involving either very large packed arrays or "touch once"
+streaming algorithms that make CPU caches near-useless.
+
+Ilya Prime:
+
+* cares about performance to exclusion of most other factors
+* needs to be able to efficiently exchange large data sets with
+  special-purpose libraries, such as audio and video codecs
+* needs to be able to efficiently work with packed data
+* doesn't care about PBC one way or the other
+
+
+=head2 "Ilya Double Prime": Intensive Threading User
+
+Ilya Prime writes high-performance threading-intensive code,
+typically involving simulations, server implementations, or
+simply efficient use of a heavily threaded CPU architecture.
+
+Ilya Prime:
+
+* cares about performance, memory footprint, thread-safe
+  correctness, and possibly security/DOS prevention APIs
+* needs to be able to create and destroy large numbers of
+  threads rapidly
+* needs to be able to transit data between threads rapidly
+* cares about PBC only inasmuch as it affects threading
+  performance and overall process memory footprint
+
+
=head2 "Magpie": Lover of shiny things

Magpie sees something shiny -- a new runtime, or a new language, or

Re:Ilya user type tweak

treefrog on 2005-07-22T14:28:19

"Ilya Double Prime" and "Treefrog" have a lot in common....

regards, treefog