DmitrySoshnikov/syntax

Tokenizer: support onToken event

DmitrySoshnikov opened this issue · 1 comments

DmitrySoshnikov commented

We should expose the onToken event, so the clients can subscribe, and route the parsing events based on the token.

yy.tokenizer.onToken = token => {
  if (token.type === 'L_PAREN') {
    ...
  }
};