V1.10.0 tokenising bug with ELSE IF

[NealeTools]

Posted on FB: https://www.facebook.com/groups/ZXNextBasic/posts/1764228040770194/?comment_id=1765746263951705

210 %z=% SPRITE OVER (j+1,1 TO 34,6,6): IF %z=0 THEN NEXT %i: RETURN: ELSE IF %z=34 THEN: ELSE DRIVER %49,%2,%59: %q=%q+1: %s=%s+1: %m[j]=%m[j]-1: %n[j]=0: %v=%v+1&7+112: SPRITE %v,%SPRITE AT (z,0),%SPRITE AT (z,1),%56,%1: SPRITE CONTINUE %v: SPRITE %z,,,,0:SPRITE %SGN{-j-1},,,,%m[j]>4113+(m[j]<5145): IF %m[j] THEN NEXT %i: RETURN: ELSE GOSUB %1000: NEXT %i: RETURN

Throws NONSENSE in BASIC on statement 6, which is the ELSE IF %z=34 part
Adding an extra space between ELSE and IF avoids the error.

This was fine on prior version of VSCode.

--repro with more details:

FAILS:
10 IF %z THEN: ELSE IF %z=34 THEN: ELSE PRINT "3"

PASSES: (extra space between ELSE and IF)
10 IF %z THEN: ELSE IF %z=34 THEN: ELSE PRINT "3"

also PASSES! (single space but single value after equals)
10 IF %z THEN: ELSE IF %z=3 THEN: ELSE PRINT "3"

And super weird:
PASSES: (If seems only values >32 fail.)
10 IF %z THEN: ELSE IF %z=32 THEN: ELSE PRINT "3"

[remy]

This looks like a great bit of detail to help me quickly get to the bottom of the issue, thank you.

I'll try to get a fix out this weekend, otherwise it'll be Monday

[remy]

Getting a release ready.