LLM generates text that the grammar does not allow
mattiasarro opened this issue · 1 comments
mattiasarro commented
When running the example code, the following exception gets thrown:
UnexpectedCharacters: No terminal matches '"' in the current parser context, at line 1 col 7
The model had generated output {"a": "
and the parser did not parse this particular output. Since {"a": "
is a valid start of a JSON object, the error is a bit confusing. But playing around with different grammar, prompts and models a bit shows that the model can (and often does) generate text the parser can not parse, which results in lark.lexer.UnexpectedCharacters
exception.
mattiasarro commented
This is actually a bug in rellm, created a PR there.