kaleidawave/ezno

Parsing bug w/ `do while` found by `parser/fuzz/module_roundtrip_structured`

Opened this issue · 3 comments

let output1 = module1.to_string(&to_string_options);
let Ok(module2) = Module::from_string(output1.to_owned(), parse_options) else {
panic!("input: `{input}`\noutput1: `{output1}`\n\nThis parse should not error because it was just parsed above");
};
strikes again.

input: `do (/a/d); while (this);
`
output1: `do /a/d; while (this);
`

It looks like the parse and print removes the parentheses from the do statement, which causes Ezno to interpret the first / as a "divide" statement.

image

Biome Playground of input: https://biomejs.dev/playground/?code=ZABvACAAKAAvAGEALwBkACkAOwAgAHcAaABpAGwAZQAgACgAdABoAGkAcwApADsACgA%3D

Biome Playground of output1: https://biomejs.dev/playground/?code=ZABvACAALwBhAC8AZAA7ACAAdwBoAGkAbABlACAAKAB0AGgAaQBzACkAOwAKAA%3D%3D

Another example I think:

input: `while (class e extends async function({ [(/a/d)] : d }) {} {}) {
}
`
output1: `while (class e extends async function ({ [/a/d]: d }) {} {}) {}
`

This parse should not error because it was just parsed above. 
error: `ParseError { reason: "Expected identifier at variable reference, found Divide", position: 42..43 }`

https://www.typescriptlang.org/play/?#code/O4CwlgNgpgBAFAYwgQwM6prKAPALlAOwBMM0BPAhGAMwFdLcwB7AuAbxgG04B6ZHogEoAujABcMIjAC+gmG2nzZ8gFDSgA

sounds like the actual JS is a bit angry at runtime, since the async fn doesn't count as a constructor, but I think it's syntactically correct. The printed version (output1) is also close to what biome and prettier output.

image

this seems relevant

Sounds like

ezno/parser/src/tokens.rs

Lines 502 to 522 in 245d530

/// Some tokens can be used as names for variables, methods (eg 'get' in `Map.get()`). This function
/// takes a [Token] and returns its name as a [String] and the location as a [Span]. Will throw [`ParseError`] if
/// cannot convert token to string
pub(crate) fn token_as_identifier(
token: Token<TSXToken, TokenStart>,
at_location: &str,
) -> crate::ParseResult<(String, Span)> {
let position = token.get_span();
let name = match token.0 {
TSXToken::Identifier(value) => value,
TSXToken::Keyword(keyword) => EnumVariantsStrings::to_str(&keyword).to_owned(),
token_type => {
return Err(ParseError::new(
crate::ParseErrors::ExpectedIdent { found: token_type, at_location },
position,
));
}
};
Ok((name, position))
}

needs to allow regex as a name for variables, or there's a bug somewhere where it initially thinks it can permit regex as names for variables