KeyError when training the Chinese dataset

Question

KeyError when training the Chinese dataset

KevinWang676 opened this issue a year ago · 4 comments

Hi, does anyone know how to fix the KeyError: ',' when training the Chinese dataset? I'm using chinese_cleaners defined here. Thank you!

Answer 1 · 2023-09-21T06:23:40.000Z

fixed in latest patch.
unavailable tokens will just be skipped.

Answer 2 · 2023-09-21T06:48:31.000Z

Hi, thank you so much for the fix! However, you may need to change the variable to cleaned_text in line 40 and define sequence = [] in line 38. I fix the typos in my folk.

Also, I wonder if missing the symbols like the comma , would affect my training process and results. Thank you!

Answer 3 · 2023-09-21T08:56:37.000Z

Hi @p0p4k, I wonder if missing some symbols like the comma , would affect my training process and results. Thank you!

Answer 4 · 2023-09-21T09:45:12.000Z

You can add missing punctuations in the punctuations list and try training with that first.