support for external lexers and partial input
mgood7123 opened this issue · 1 comments
could mpc be used with external lexers (as in an actual proper lexer such as gnu lex/flex), where tokes are fed to the parser, optionally accepting a token type, if so is lookahead supported and if so how far can it look ahead and can it backtrack if a lookahead does not end up matching, and does it support a mpca_lang or similar
for example:
while(token = get_next_token()) parse(token.token, token.type);
given the rule mpc_string("abc");
and given get_next_token()
returns the following
token.token = "a"
token.type = TOK_CHAR
token.token = "bc"
token.type = TOK_STR
could this partial input match the rule mpc_string("abc");
without causing the rule to terminate due to parse reading EOF
in its input from the string "a"
due to "a"
by itself not matching the rule mpc_string("abc");
resulting in parse()
to error
You could maybe do this, although you would need to make sure the parsers you use take your custom token type as input and produce the AST as output. Also the back-tracking wouldn't be able to interact with the token parsing I.E. putting tokens back into the stream but honestly at this point it seems like it would be better to write the parsing function by hand.