genco tokenization of rust lifetimes broken in 0.16

Question

genco tokenization of rust lifetimes broken in 0.16

dvc94ch opened this issue 3 years ago · 4 comments

So it's nice that it builds on stable, but it doesn't quite work with lifetimes. ' static is not valid rust, rustc complains that a "character literal may only contain one codepoint".

Answer 1 · 2022-03-20T00:19:09.000Z

There might not be enough info to resolve this. Apparently the ' punctuation might not have joined spacing. But I'll look into it to see if there's anything that can be done.

Cheers!

Answer 2 · 2022-03-25T06:10:23.000Z

Should be fixed now in 0.16.1!

Answer 3 · 2022-03-27T12:35:30.000Z

Gave it a try. The fix fixes the life time issue but found another one. Identifiers are split in weird ways: tmp0 _0 should be tmp0_0.

Answer 4 · 2022-03-28T21:16:47.000Z

Yeah. I don't think there's much to be done there unfortunately. We can't tell when there is some space between two things and no space.

To construct an identifier that is concatenated we'd have to resort to some form of markup or formatting the identifier inline, like: #(format!("{}{}", "tmp0", "_0")).