The problem with my previous implementation of nested [% IF expr %] is that the nature of [% FOREACH item IN list %] forcefully requires shallow-to-deep resolution.
The negative-lookahead solution only allows for deep-to-shallow resolution, and there isn't really any good way to change that.
So it appears that I've gone as far as I can in memory-conservation mode, and it's time to spend some of my remaining budget.
The real problem here, as I mentioned earlier, is the [% END %] tag. By using the same closing tag for everything it's impossible to match an opening tag to the appropriate closing tag.
Changing to directive-specific closing tags doesn't really help either because it doesn't fix the nested scenario of the same tag. So switching to a reverse-style [% IF expr %]foo[% FI %] won't get us anywhere either (it will just create the appearance of doing so).
What would be REALLY nice would be to have every single tag pair different. All we need to do is invent our own TT derivative. We'd never expose this alternative TT language to the users, just use it as an intermediate format.
Since we know that the negative-lookahead method can identify all the tag pairs, this turns out to be quite feasible (if somewhat expensive in memory terms).
To start with, we redefine the lookahead search slightly to match the longer FOREACH tags as well.
my $PREPARSE = qr/
$LEFT ( IF | UNLESS | FOREACH ) \s+
(
(?: \S+ \s+ IN \s+ )?
\S+ )
$RIGHT
(?!
.*?
$LEFT (?: IF | UNLESS | FOREACH ) \b
)
( .*? )
(?:
$LEFT ELSE $RIGHT
(?!
.*?
$LEFT (?: IF | UNLESS | FOREACH ) \b
)
( .+? )
)?
$LEFT END $RIGHT
/xs;
We can use this to iterate over the string and create a set of custom tags for each different match. At the same time, we might as well make those custom tags based on a single letter (regex tricks get much easier when string elements are only one character long).
The only complicating factor is that if we use the string-chomping left and right brace matches now, we'll need to be careful not to chomp again on the second pass.
my $id = 0;
1 while $copy =~ s/
$PREPARSE
/
my $tag = substr($1, 0, 1) . ++$id;
"\[\% $tag $2 \%\]$3\[\% $tag \%\]"
. (defined($4) ? "$4\[\% $tag \%\]" : '');
/gsex;
The result of this is that we take a template that looks like this...
People:
[% FOREACH item IN list %]
[%- item.name %] <[% item.email %]>
[% END -%]
Cool People:
[% FOREACH item IN list %]
[%- IF item.cool %]
[%- item.name %]
[% END %]
[%- END -%]
and turns it into this...
People [% foo %]:
[% F3 item IN list %]
[%- item.name %] <[% item.email %]>
[% F3 %]Cool People:
[% F2 item IN list %][% I1 item.cool %]
[%- item.name %]
[% I1 %][% F2 %]Done!
The price for this preparsing step isn't cheap, it chews up a whopping 15k without adding any new features (which is more than we paid for UNLESS, ELSE and IF recursion combined).
But with this one, we can switch to top-to-bottom and shallow-to-deep processing. And we can also do recusive top down processing, with some simple stash localisation so that "item" in [% FOREACH item IN list %] is resolvable.
my $CONDITION = qr/
\[\%\s
( ([IUF])\d+ ) \s+
(?:
([a-z]\w*) \s+ IN \s+
)?
( $EXPR )
\s\%\]
( .*? )
(?:
\[\%\s \1 \s\%\]
( .+? )
)?
\[\%\s \1 \s\%\]
/xs;
This single match approach means that, bizarely, both the following are legal in T:Tiny.
1 while $text =~ s/
$CONDITION
/
($2 eq 'F')
? $self->_foreach($stash, $3, $4, $5)
: eval {
$2 eq 'U'
xor
!! # Force boolification
$self->_expression($stash, $4)
}
? $self->_process($stash, $5)
: $self->_process($stash, $6)
/gsex;
[% FOREACH item IN list %]
foo
[% ELSE %]
bar
[% END %]
Well, maybe not legal. But they won't throw an error, which is tolerable at least.
[% IF foo IN bar %]
[% END %]
We'll also need to spend about another 5k to factor out the code into 3 or 4 different methods, so that everything runs properly when evaluated recursively.
After all this overhead, and about 5-10k on adding the actual support for FOREACH iteration, the result is we can now arbitrarily nest IF/UNLESS/ELSE/FOREACH inside each other.
This leaves me with 68k spent out of my original budget of 100k of memory.
And without support for files yet (and thus things like PROCESS or INSERT or INCLUDE) I still have a lot to do...
