The Perl 6 design team met by phone on 23 September 2009. Larry, Patrick,
Jerry, and Nicholas attended.
Larry:
- thinking about longest token matching, and how to do it with a real DFA
- profiling standard grammar, to see where the current bottlenecks are
- been in IRC design discussions with Patrick and others
- helping diakopter with his JavaScript backend
- originally inclined to write off David Green's suggestion about ranges and non ranges, but the more I thought about it, and how series operators worked, the more I liked a variation of his idea
- that's what I spec'd
- now ranges objects are primarily objects that reflect an interval, min and max; no from, to or mutable
- Can use them to make an iterator, but you only get an iterator by ones, or
a, b, c, d
- series operator now extended to handle the notion of steps and limits
- since it can do that now, and since I always thought that
:by was ugly, it's now deprecated
- simplified the matching of alpha ranges. Used to do fancy footwork to ensure that the range would never exceed the length of the right hand argument, but now it just goes and produces an infinite lazy list if you didn't produce a valid end point that can be compared
- with simplification of ranges, looked at the interaction of ranges and subscripts in Synopsis 9
- if a range overlaps the beginning or end of an array, it throws away the part that doesn't apply
- doesn't matter if you say
0..* or 0..*-1 (or 0..Inf), they all work out the same
- looked at some spec'd feature that others came up with, negative subscripts causing unshift behaviour, and I thought that that would be too error prone, so I took that out
- spec'd a way to declare modular subscripts, so that they wrap around at both ends, just the way APL subscripts work, as it happens
- actual hacking, most was for diakopter, to fix up things that were missing
- it's now easy to pull arguments out of operators from the AST
- previously you needed to know where in the linear sequences it was
- for example, infix [operators ?] you had to know that it was first and third, and that was kind of bogus
- the string nibbler had a bogus AST node, so I fixed that up
- STD didn't do bracketing quotes right, so I fixed that
- it did not correctly create nodes from the operator precedence parser in the AST so I fixed that
- put in an optimisation that string nibbler, if it gets a null string between two things that are interpolating, throws it away, because there's no point in concatenating a null string
- improving STD error messages
- it used not to complain if you had a symbol in a block and referred to an outer lexical, but you redefined that. It's spec'd to be an error, but it didn't catch it. Now it does. It's the longest error message of anything (three lines)
- STD was giving a bad error message if you left out the space before an infix < because it would misinterpret it as a string subscript
- now give a good error message on that, if it looks like it's not intended to be a string subscript
- there was a way to backtrack out of the radix syntax, such as a
:3 without a trinary number after
- it would give a strange error message; fixed that
- there was other stuff, but it's all in the noise
Patrick:
- much of week and weekend was for my other job, so didn't get onto Perl 6, Parrot or Rakudo until Monday
- looking at refactoring grammar engine to do protoregexes, and to get closer to how the standard grammar is doing. it. Reading it,
cursor, gimme5, to do similar things for implementing the grammar engine
- it's going well
- I expect actual code by this time next week
Jerry:
- released Rakudo #21 "Seattle", named for Seattle Perl User Group
- one of the questions I got was "is this in Debian?"
- I believe nobody is packaging it for Debian yet, but I wondered if there has been any push towards Debian
Patrick:
- there are RPM packages already. I know people were working on Debian packages, but I don't know where it ended up.
Jerry:
- release went very well, very smooth
- anyone can do it, which I think I have proven
- I'd not been building Rakudo, let alone writing source for it, and yet it only took a couple of hours, must of which was waiting for the spec test results (multiple times), so kudos for the release process
Patrick:
- it works great for Parrot, so I just stole it and adapted it for Rakudo
Jerry:
- trying to keep up with what's in the mailing list and the IRC channels, and occasionally give a semi-enlightened comment about a design issue
- no blockers for me
- why a proper DFA, Larry?
Larry:
- hopefully to run faster
- profiling suggests it would help some, in the cursor implementation, but there's a lot of overhead distributed in a lot of other places
- want to avoid repeatedly running a lot of patterns, if really a DFA or a parallel NFA, instead of faking it with a trie structure for constant strings, and for non-constants regular Perl 5 patterns, and sorting them into longest to shortest order
- not sure how much it will get, but I'd like to have the "correct" algorithm there.
Patrick:
- I plan to start with the "Cheating" algorithm, or various cheats, instead of going directly to a DFA
Larry:
- for the profiling, I get a lot of overhead from running an interpreter on top of an interpreter -- it's always going to be slow.
Patrick:
- curious how performance changes in the new engine, with the redesign
- I suspect it will improve, but I don't know by how much
- I don't have a really fast regex engine underneath
Larry:
- like just delegating character classes off to Perl 5
Patrick:
- I've thought of creating one
Larry:
Patrick:
- considered implementing one using standard traditional syntax but haven't quite got there yet
Jerry:
- Patrick, you've been talking about more near-term goals and your Hague grant
Patrick:
- updated the Roadmap in August
- it's due for another update
- I have a Hague grant due to reimburse the costs for Lisbon, but I promised for that to have a much more detailed plan for Rakudo Star
- the roadmap shows that the critical components all depend on two things
- one is the grammar engine
- one is calling conventions
- almost half block on the grammar engine
- every hour spent on the grammar engine is related to Rakudo Star
- if it doesn't get done, we're going to miss our deadline
Jerry:
- you're in the early stages of PGE
- do you see any of it as parallelizable?
Patrick:
- should be parallelisable; Carl was planning to following closely
- looking at code that Cursor is using, can write many of the parts in NQP (not PIR), in particular, the operator precedence parser
- will rewrite STD in NQP to provide that to all the higher level languages
- the code changes slightly from STD, as NQP only does binding, not assignment
- some function calls will become methods, because NQP prefers methods
- when it's done, STD may be able to adopt this
- Carl's been looking at the existing code: "oh, it makes sense"
- just in time for me to change it all around
Jerry:
- progress from last time you created a grammar engine
Patrick:
- there's a good chance it may not be called PGE
- to avoid deprecation issues, it might be easier to leave the old one alone
- the new one is NQP, as NQP has regexes
- it's the same language for the parser and the action methods
- the underlying engine gets a new name, as it's pretty different
