Perl 6 Design Minutes for 05 November 2008

chromatic on 2008-11-07T20:39:49

The Perl 6 design team met by phone on 05 November 2008. Larry, Allison, Patrick, Jerry, Will, Nicholas, and chromatic attended.

Larry:

the specs officially define a Nil type that is the meaning of ()
an undef that's undef in item context
otherwise, just a null list
just to clarify the semantics of what it means to say return;
answered Patrick's email about autothreading of junctions over method calls
I think that doesn't have much impact on specs
we can clarify if necessary
in terms of hackery, I've done a lot of work on cursor and STD and gimme5
optimizations and simplifications
preparing to get rid of TRE in the lexer
threw a lot of stuff out of cursors themselves
they no longer track the original string or positional memos in the Perl 5 version
nor do they distinguish a to attribute anymore
an unbound cursor only as a position
a bound cursor has a from
for ratchety kind of matching, all it does is modify the position in a mutable cursor
rather than copying cursors, which is fairly expensive
a little more efficient
made the tryfile syntax checker work on multiple test files
keeps it from having to recompile the standard grammar on every test
it does leak rather badly
you can't parse the entire test suite with one invocation, unless you have a lot of virtual memory
refactoring the various methods which determine cursors to try to make decisions earlier
don't have to make extra function calls to make a decision which should be made at a higher level
some speed gains, some preparation for lexer changes
another preparation is to start emulating NFG semantics
when I set up for the original parse, I now do an unpack "U*" as an emulation of NFG
one integer per character
some of my tests in my regex emulation now test against that array
rather than against the original string
it runs no slower, even accounting for the conversion
eventually the lexing I want to do with a homebrew DFA matcher will use that integer array directly
probably also buys some efficiency
Perl 5 is not very efficient about recalculating positions
it only caches a few UTF-8 positions for a string
if I'm bouncing back and forth between the beginnings and endings of tokens, it has to keep recalculating UTF-8 from one direction or the other
the net NFG will be a win
even if some tests need to use the string version
still planning to get rid of TRE
the workarounds cause the lexer to take half a GB of disk space
hope it'll run faster too

Patrick:

worked on exceptions in Parrot
mostly identifying places where the spec needs work, or how they don't work as I expect them
answering questions about the specification
worked on lexical variable handling in Parrot
tomorrow, Jonathan and I will switch over to the new container/value semantics so they work properly

does that fix list assignment?

Patrick:

it's necessary, but not sufficient
PCT also needs some changes

Jerry:

worked on some refactoring of Rakudo's action grammar
action methods
there were many different types of calls into the metaobject
it was quite a mess
I implemented an isa method on PCT's Node class
converted Rakudo to use that instead of .WHAT or .HOW.isa
definitely more refactoring to do in the action methods
I'll attack that as I can
Jonathan made some changes to Rakudo's use
instead of compile time, it happens at INIT time
precompiled modules work now
but it's still a hack
use shouldn't be done at INIT time
his changes necessitated changes to my import patch

Patrick:

that was the case anyway

Jerry:

I knew that was the case
I just need the time to rework that
still preparing for the Parrot Developer Summit
again, that will continue until after it happens

Will:

did some Parrot work this week
fixed up the sub-level tracing that's been broken for some of the more complex code
turned out to be a cut-and-paste job from something Jonathan had done for Rakudo in Perl 6
trying to go through the deprecated list, getting rid of some low-hanging fruit
deleted quite a bit of code this week
looking at the tracing stuff to help me figure out where Tcl is spending most of its time
we seem to be invoking PGE a lot
looking forward to some of the work Patrick has on his plate
hopefully that'll get Tcl and Perl 6 running faster

I fixed some bugs
fixed some crash bugs
found some new crash bugs and fixed them
also trying to remove deprecated code
tracking down some other crash bugs, but they're tricky
cleaning up code is a priority there
still thinking about how to do PIR-level profiling
we sorely need it

Nicholas:

the release candidate didn't get out last weekend
still might not get out this weekend
waiting on a CPAN release of File::Path

Patrick:

remember that there are two init times
there's the semantics of Perl 6 INIT block
there's the Parrot sub :init as well

Jerry:

I meant the Perl 6 version

Patrick:

there's nothing prior to :init in Parrot terms, if you're talking about Parrot
we have to keep that in mind

Jerry:

Jonathan pushes onto $?INIT in the Perl 6 terms

Patrick:

it has to happen earlier than that

Nicholas:

Larry, what version of Perl are you using, in terms of UTF-8 slowness?

Larry:

5.10
it does some caching of pos
but if your lexer runs back and forth to test the same thing various times, it still has to do some stuff

Nicholas:

it ought to be able to store three or more positions
might be as simple as a #define
I haven't done that

Larry:

I'm moving away from the whole UTF-8 model anyway toward the NFG model
that unpack is amazingly fast
I thought I'd lose some time, but it was just as fast
maybe that's some kind of indication that we're going the right direction
I'm not actually doing NFG, just assuming I'm in NFC form
NFC is a subset of NFG

Jerry:

with this approach, will you always need a hybrid solution?

Larry:

it depends on what your VM is

we can recommend one for you

Larry:

some things are easier to check in Perl 5 with the string matching
no decent way I know of to ask whether a particular character number matches a particular character class
I'd have to convert it to a string, then do a pattern match against it
as far as I know
unless you're aware of a way in which of saying "Is character #582 an alphanumeric?"

Nicholas:

I assume you can hijack UTF-8 swatch caches to figure that out in Perl 5

Larry:

it would take an XS hack right now
don't think those are exposed at the integer level

Nicholas:

part of that is implemented as Perl 5
converted to XS, at least the slowest parts

Larry:

I doubt it would be difficult to make an API for direct access in a version of Perl
it wouldn't be in the current version

Nicholas:

not likely to be sufficiently faster, when you account for developer time

Larry:

essentially, I'm matching against UTF-32

Patrick:

do we have a good idea or list of what methods are defined on junctions?

Larry:

there's probably a secret method name which does the autothreading, which Object delegates to
other than that, maybe some methods to extract out the bits of it
they would not have to be human readable
I'd de-Huffmanize those names to prevent accidentally running into them

Jerry:

this came up with regard to any container, not just junctions
how do you call a method on a container, not something that hypers over the values?

Patrick:

it's not automatic, as it is with junctions
the method that kept coming up in my mind is .values to grab the values out of the junction

Larry:

there'll be a method to do that, but not with that name
.junction_values
something unlikely to collide with anything else

Patrick:

there was a discussion about stringification being made special somehow
the printed stringification and default stringification might be different
the .perl method on a junction might return a list of strings, rather than a string
how do we get a string that contains the junction operator in the middle, rather than a junction of two strings
has that made it into Synopses somehow?

Larry:

a multimethod which converts a junction to a string will fire directly without autothreading
Str(Junction)

Patrick:

it's a method, not a function?

Larry:

it doesn't really matter, as long as it's defined within Junction

Patrick:

Junction.Str always gets that version, and does an autothread?

Larry:

if a Str method is defined in Junction then it won't autothread
only autothreads if Object gets the method dispatch.


always produces a string which looks as if it came from .perl
everything falls out of the current mechanism
we don't have to define special cases



Patrick:


one more question
what is the syntax for matching a string or object against a specific rule within a grammar?
a lot of people have smart matched against the name of the rule


Larry:


underneath, you create a match
then just call a method on that
the method is the rule
how much syntactic sugar we want to wrap around that, I'm not sure


Patrick:


can I tell people that they're doing it wrong?


Larry:


it probably is, yes
it sounds wrong, without seeing it


Patrick:


essentially $variable ~~ Gramar::Rule_Name


Larry:


that seems unlikely to work without someone deciding to create the match
it's possible the smart match operator could be smart enough to do that


Patrick:


if I do $variable ~~ sub


Larry:


that depends on the arity of the sub
similar distinction
"Oh, this is actually a method"
presumably of zero arity, except for the invocant
essentialy the string has to be passed in as the invocant


Patrick:


that is one of those things where the smart match is macro-like
it recognizes that it's a sub, not a list operator


Larry:


it recognizes a routine of some sort
it needs to know that it's a method from just inspection
just naming a method doesn't give it enough information
unless you put it in method form...


Patrick:


leave methods out of it for now
does that syntax check the arity of a sub against something
or does it treat it as a list op?


Larry:


it'll call the sub with the left-hand argument as the first argument


Jerry:


as opposed to $variable ~~ &sub


Patrick:


normally a sub name in a term position is a listop


Larry:


seems like it still is
I don't think it'll work without the ampersand
you have to allow a general expression on the right to return a value to smart match
you need explicit curlies or the ampersand


Jerry:


in the recent past, I remember discussions with the PPI folks that Perl 6 can't parse Perl 6 any better than Perl 5 can parse Perl 5
although things aren't perfect with the parse yet or STD
and we aren't using BEGIN and macros yet
azawawi has created a syntax highlighter based on STD
it's really cool
feather has syntax-highlighted versions of all of the spec tests
they're linked from spec.pugscode.org
you can see syntax-highlighted Perl 6 today
hasn't been reported anywhere
definitely worth mentioning
effectively a built-in







syntax highlighted examples
Eric Wilhelm on 2008-11-15T18:37:00
jerry:

    azawawi has created a syntax highlighter based on STD

    it's really cool

    feather has syntax-highlighted versions of all of the spec tests

    they're linked from spec.pugscode.org

    you can see syntax-highlighted Perl 6 today
This made me go look at http://spec.pugscode.org/ but I didn't see anything there, so I had to ask about it.  The highlighted examples are accessible within the texts of the synopses (e.g. http://perlcabal.org/syn/S02.html), where you'll find links like "Highlighted: small|full", which give you either an inline view or a new window (respectively.)  Also see http://feather.perl6.nl/user/azawawi/html/.