CustomScanner Unexpected EOF

[DominikGri]

Hi,
i uses the Custom Scanner "AlphabetScanner" from the documentation.

var Parser = require("jison").Parser;

function AlphabetScanner() {
 /* ... */
};

var parser = new Parser(grammar);
parser.lexer = new AlphabetScanner();
console.log('parse: ', parser.parse('Teststring'));

But i always get "Error: Parse error on line NaN: Unexpected 'EOF'"!

[GerHobbelt]

Which jison version are you using?

[DominikGri]

Version ^0.4.18

[GerHobbelt]

If you try this with jison-gho, it works. (npm: https://www.npmjs.com/package/jison-gho, github: https://github.com/GerHobbelt/jison)

Note that the jison-gho fork can be more strict when it comes to custom lexers: see that package's examples:

• examples/documentation--custom-lexer-ULcase.jison showcases the preferred way in jison-gho by writing a custom lexer inside the %lex.../lex section: here the lexer instance MUST be named lexer and adhere to a certain minimal interface, which is verified by jison-gho.
• examples/documentation--custom-lexer-ULcase-alt.jison showcases the classic way in jison/jison-gho by having the custom lexer dropped in the tailing %% code section. Then there are no requirements checked and you can assign it to parser.lexer any way you like to produce a working parser+lexer combination.

---

If OTOH you wish to work with vanilla jison (0.4.x), you might want to go and code-inspect and/or debug=step through the generated parser source code.

Two things jump out from your OP:

1. NaN for the line number is probably due to the custom lexer not tracking a yylineno member value (insert handwave as I haven't checked vanilla parser kernel code whether it uses the yylloc or yylineno!)

2. 'unexpected EOF' MAY also be due to a mismatch between your grammar and the lexed input: if your grammar doesn't expect an EOF when the lexer spits one out, this is exactly the error message to expect.

[GerHobbelt]

(See the jison-gho example jison files mentioned above for a working sample grammar which should accept your test input ("Teststring"))