sweet-js/sweet-core

Tokenizer algorithm will fail on a regexp after contextual `of` keyword

Opened this issue · 3 comments

marijnh commented

I.e. in code like for (var a of /b/) {}. Related to acornjs/acorn#525 , since Acorn's tokenizer regexp disambiguation is based on Sweet.js' approach.

gabejohnson commented

This isn't an issue with contextual keywords in general.
https://github.com/sweet-js/sweet.js/blob/616f4dfc0c15fed59e1d0513ab096a5606803b74/src/reader/utils.js#L556
checks to see if the preceding token is a keyword.

I think this is an issue with of in particular. It's not in the list of keyword tokens.

Edit: it is an issue with contextual keywords.

disnet commented

It's not in the list of keyword tokens.

That's because of isn't a keyword. It's contextual :)

I'm pretty sure yield is another one we have a problem with for the same reason. I've been meaning to look at yield for a while (as this todo attests). Probably await too but we don't support that at all yet.

gabejohnson commented

It's not in the list of keyword tokens.

It must have be late when I wrote that. I use Array.of frequently and have implemented of methods a handful of times.