bakhbyergyen opened this issue 3 years ago · 0 comments
hi, I wanted to know that, why zh and ja datasets are split by character? not word by word? when building a dataset, sentences can be split by words, not characters? thank you.