HOP::Lexer 0.02

Ovid on 2006-01-27T20:18:38

I've uploaded a new version of HOP::Lexer. In "Higher Order Perl", the lexer assumes that you'll be passing an iterator as the first argument to make_lexer. That means if you're trying to lex a single block of text, you have to do this:

my @text  = $text;
my $iter  = sub { shift @text };
my $lexer = make_lexer( $iter, @tokens );

However, lexing a single block of text was common enough that I provided this:

my $lexer = string_lexer( $string, @tokens );

I thought about overloading make_lexer to check if the first argument was a subref or a scalar, but I felt that this might cause strange bugs if people didn't know the difference. I'm probably overly paranoid.

For more information about using a lexer, see my perl.com article.