ualberta-smr/varclang

Backtrack stack imbalance when #if/#ifdefs have conditions that get normalized to True

Closed this issue · 0 comments

The following test case fails an assertion because I try to call
CommitBacktrackedTokens when no backtrack positions have been set.

Example:

#define A

// This ifdef gets normalized to "True"
#ifdef A
int i;
#endif

The #ifdef A gets assigned the normalized presence condition True since there
is a #define A in context True. The lexer inserts a split token at this
#ifdef but the parser doesn't enable a backtrack position. This is because the
parser compares the current parser condition (True) to the condition of the
split token (which was normalized to True) and sees that a split is not
neccessary.

This check was originally added to fix issue #14 so we can't simply remove it
and set a backtrack position at every split token.

The assertion in CommitBacktrackedTokens is failing when it is called inside
Parser::ParseExternalDeclaration. When parsing external declarations, the
code currently consumes all split tokens and calls CommitBacktrackedTokens
for each split token. We never had a problem with split tokens not having a
backtrack position for external declarations before since before normalization
was implemented, an #ifdef couldn't have presence condition True. Note that
when parsing external declarations, the parser condition is always true (since
the parser condition is only used when performing backtracking inside a
function body).

One way which I can think of to fix this issue is to only insert split tokens
when backtracking will be neccessary. This will make things cleaner in the
parser.