summaryrefslogtreecommitdiff
path: root/spec/unit/pops/parser/lexer2_spec.rb
AgeCommit message (Collapse)AuthorFilesLines
2014-06-09(PUP-2693) Fix length calculation for sq-stringHenrik Lindberg1-0/+10
The calculation of string length was done wrong for dingle quoted string resulting in negative lengths.
2014-05-16Merge remote-tracking branch 'upstream/stable'Josh Partlow1-2/+10
* upstream/stable: (PUP-2568) Remove unnecessary calls (maint) Reorder code to increase clarity (PUP-2581) Make illegal names interchangeable to legal variable work (PUP-2568) Downcase class name before validating Conflicts: lib/puppet/pops/parser/eparser.rb Rebuilt eparser.
2014-05-16(PUP-2581) Make illegal names interchangeable to legal variable workHenrik Lindberg1-2/+10
In interpolation when using expression mode ${} and having an illegal name (e.g. _x) that resulted in a syntax error as the _x got translated to a STRING token (a bare word not being a name). The interpolation logic could then not transform it into a (valid) variable expression (i.e. $_x). The change is to let the lexer emit a WORD token for the illegal NAME. The interpolation then changes that to a VARIBLE if it is the only expression. If the WORD token escapes into the grammar, it is equivalent to having used a quoted string in the source text. Note that WORD only accepts unqualified input, anything with a : that is not a valid NAME is still flagged as an error. Thus _x::y, x::_y are both illegal NAME.
2014-05-07(PUP-2288) Make [ be LISTSTART if first in file or preceded by WSHenrik Lindberg1-4/+11
The problem was that there was only one special case being handled by the lexer to dis-ambiguate if an expression followed by [ meant the start of an access operations (e.g. getting an entry out of an array), or if it represented the start of a literal array. The supported special case was NAME followed by non-WS followed by [. This meant that cases like foo(3) [a,b,c] were always treated as an access operation. The fix generalizes the dis-ambiguation so that all access operations must be written without interleaved WS.
2014-04-08(PUP-2129) Fix issues with false interpolation start (no-such-method)Henrik Lindberg1-0/+11
This is a fix for the reported issue that 'slurpfunc' is an unknown method. Thankfuly that problem masked a much worse problem in that any false interolation start e.g. $%a was not correctly implemented, and there were no tests that covered this. The same issue also existed for interpolation in heredoc. This commit fixes the interpolation problem, and adds tests.
2014-03-08(PUP-1897) Fix failing lexer2 test, and update with additional testsHenrik Lindberg1-0/+15
Tests were failing due to additional token marking the end of an EPP render expression.
2014-03-06(PUP-30) Make epp parser emit EPP_START, and generate ProgramHenrik Lindberg1-1/+6
Parsing EPP now results in a Program model (if producing only an EppExpression there are no source code references). The EPP_START tokten was not emitted (and its precence was not tested for).
2014-03-06(PUP-30) Complete implementation of sublocated expressionHenrik Lindberg1-1/+2
This completes the implementation of SubLocatedExpression. The SubLocator refactored to Locator (instead of being nested inside the support for Heredoc). Heredoc lexing, and the grammar now cooperate to ensure that sublocated model is wrapped.
2014-02-28(PUP-1814) Make double backslash in sq string behave as documented.Henrik Lindberg1-1/+8
This makes it possible to escapce a backslash in a sq string. Previously this did not work which made it impossible to end a sq string with a backslash.
2014-02-27(PUP-1576) Make hyphen in bare words legal (future parser)Henrik Lindberg1-0/+18
This adds the ability to use a hyphen in a bare word. The result is the same as if the bare word had been quoted in single quotes. It is not allowed to start or end a bare word with a hyphen, those are always interpreted as :MINUS.
2013-12-05(#22363) Fix problem with integer string to Integer on Ruby 1.8.7Henrik Lindberg1-0/+10
The problem was caused by the logic using the Kernel method Integer(string, radix) which was introduced in Ruby 1.9.2. We do not need to pass radix, since the string converted starts with 0, 0x, or 0X in all cases.
2013-12-05(#21874) Apply making name[x] and name [x] different.Henrik Lindberg1-1/+9
This also includes a cleanup of LAMBDA (unused/dead logic in grammar)
2013-12-05(perf) Move all behcmarks to one unit test and exclude it with filterHenrik Lindberg1-25/+0
All parser benchmarks are now located in one spec unit test. Thy are automatically excluded by a filter that can be turned off by setting the ENV variable BENCHMARK to true when running the tests.
2013-12-05(#22363) Add parse / validate / evaluate tests, fix evaluator issuesHenrik Lindberg1-1/+0
This adds a comprehensive parse/validate/evaluate test for the future evaluator. Several parsing and evaluation issues were found and corrected.
2013-12-05(perf) Make future parser use new faster lexerHenrik Lindberg1-5/+9
This makes the future parser use the new faster lexer. As a consequence features provided by the old lexer has now moved to the parser (keeping track of namestack). Locator information is still not optimized wrt the parser as the tokens/locator information are adapter to the "old" way (room for performance improvements). Tests are updated as the new lexer is both more restrictive, and in some case more relaxed (where regular expressions may occur).
2013-12-05(maint) Move Lexer2 to the correct namespace (Puppet::Pops::Parser)Henrik Lindberg1-2/+2
It was named Lexer2 in the gloval namespace.
2013-12-05(#20516) Add support for epp templates in new lexerHenrik Lindberg1-6/+88
This adds support for epp templates to the new Lexer2 without affecting the overall performance of the lexer. This does not add the full epp template feature, only the support in the lexer and the EppParser that shows how the lexer is used to perform epp lexing when parsing. This commit also refacores method names for error handling since the most common way to report a lexer error is to add location information.
2013-12-05(perf) Add Lexer2 - a faster Lexer replacementHenrik Lindberg1-0/+304
The Lexer2 (not yet used by the parser) performs >3x faster on lexing of 10 lines of mixed code including simple expression interpolation than the original lexer (which does not keep track of detailed positions).