lezer-parser/lezer

generator creates invalid code

BryanCrotaz opened this issue · 5 comments

I have written a simple grammar:

@top Program { expression }

expression { Name | Number | BinaryExpression }

BinaryExpression { "(" expression ("+" | "-") expression ")" }

@tokens {
  Name { @asciiLetter+ }
  Number { @digit+ }
}

then generated the parser;

lezer-generator ./test.grammar -o ./test.ts

and get

// This file was generated by lezer-generator. You probably shouldn't edit it.
import { LRParser } from '@lezer/lr';
export const parser = LRParser.deserialize({
  version: 14,
  states:
    "!QOQOPOOOQOPO'#C`OOOO'#Cb'#CbQOOOOOO]OPO,58zOQOPO1G.fOeOPO7+$QOOOO<<Gl<<Gl",
  stateData: 'j~OQQORQOVPO~OWTOXTO~OYVO~O',
  goto: 'fVPPPPWP]VQOPTQROQSPRUT',
  nodeNames: '⚠ Program Name Number BinaryExpression',
  maxTerm: 10,
  skippedNodes: [0],
  repeatNodeCount: 0,
  tokenData:
    '!a~RVxyhyzm{|r}!Ow!Q![|!c!}!U#T#o!U~mOV~~rOY~~wOW~~|OX~~!RPR~!Q![|~!ZQQ~!c!}!U#T#o!U',
  tokenizers: [0],
  topRules: { Program: [0, 1] },
  tokenPrec: 0,
});

Error: Property 'deserialize' does not exist on type 'typeof LRParser'.

I've searched the code base and deserialize doesn't appear in any of the parser source. Looks like the generator is out of date compared to the parser.

Further investigation discovered your typescript definitions are missing the static deserialize:

// parse.d.ts

...
export declare class LRParser extends Parser {
  // add this line and create a type for spec as there isn't one defined
    static deserialize(spec: any): LRParser;

...

That's valid code, as in it works, it's just not TypeScript code. You're not doing anything wrong except that you're running TypeScript on it.

You publish typescript types, but they're wrong.

No, they just don't show internals that the parser generator output uses.

It's not internal if it ends up in the generated project file. That file is part of the app, so it will be type checked during the app build.

So that's public api not private.