icerpc/slicec

Make our Lexers Infallible

Opened this issue · 0 comments

LALRPOP's error recovery logic only allows it to recover from errors originating in LALRPOP.
So, if the lexer returns an error (say because a user typed '%', an unknown symbol in Slice) we would be incapable of recovering from it.

Instead, we should add an extra 'error' token to the lexers, and return Ok(error_token) to signal a lexer error, rather than Err(...). This will allow LALRPOP to gracefully recover from such lexer failures.