Redo GN parser to make operator precedence work
And adds more tests for related things.
The top-level for blocks/ifs is still recursive descent, the expression
parser is a Pratt parser (see e.g.
http://javascript.crockford.com/tdop/tdop.html). The basic idea there
is to have the precedences (and associativities) encoded relative to each
other and only parse up until you hit something of that precedence.
There's a dispatch table in expressions_ at the top of parser.cc that
describes how each token dispatches if it's seen as either a prefix or
infix operator, and if it's infix, what its precedence is.
I changed the tokenizer a bit to return more specific tokens to
distinguish generic identifiers from keywords, and generic operators from
specific PLUS_EQUALS (for example). I discovered that \n is an important
part of the grammar (it seems like it's necessary to determine the end of
expressions inside blocks) so I also added a NEWLINE token to the tokenizer
to let the parser deal with that cleanly.
It's only tested so far on the unittests so it's quite plausible that I've
broken stuff. Anything in particular I should try running it against?
[email protected]
Review URL: https://chromiumcodereview.appspot.com/22481008
git-svn-id: svn://svn.chromium.org/chrome/trunk/src@217871 0039d316-1c4b-4281-b951-d872f2087c98
10 files changed
