karminski/pineapple

Behavior and EBNF mismatch regarding `Ignored`

Closed this issue · 2 comments

KevinXuxuxu commented

e.g. The following pineapple source code should raise lexer error (notice the whitespace between $ and a), but still passed:

$ a = "pen pineapple apple pen."
print($a)

More generally speaking, Ignored in EBNF does not appear between every pair of other tokens, so should not be skipped in lexer whenever seen. Instead, it's better to treat Ignored as an individual token so that it can be correctly parsed.

karminski commented

Thanks, I will fix it later.

karminski commented

FIXED, see 23cee0a.

Added the LookAheadAndSkip() function for check and skip Ignored token.

  • 👍1