Shift JIS decoding error

Question

Closed this issue 2 years ago · 4 comments

THTK version: 12

The ECL compiler will generate error messages during processing a Shift JIS character '能' (0x945c in Shift JIS)
Here is the error messages:

st06bs.decl:806,0: unterminated string
st06bs.decl:806,0: syntax error, unexpected end of file

Here are the lines in st06bs.decl:

ins_537(60, 3300, 0, "能");
ins_401(60, 4, 0.0f, 128.0f);

It may be caused by multibyte character decoding.

Answer 1 · 2022-03-27T16:25:14.000Z

Which text editor are you using? Make sure the text files is encoded to shift-jis.

Answer 2 · 2022-03-27T16:50:30.000Z

Workaround: use escape sequences :-)

Answer 3 · 2022-03-27T17:07:39.000Z

Oh wait, 5Ch is \, lol...

Answer 4 · 2022-03-27T17:53:29.000Z

There are three ways to fix this

somehow use mblen to ignore multi-byte sequences in strings
same as 1, but roll our own mblen that doesn't depend on the libc locale.
add a special flag to do utf-8 <-> shift-jis conversions after then strings are read

Only the last option makes sense to me.