tihu-nlp/tihu

Merging Tokens and Punctuation files

Opened this issue · 0 comments

b00f commented

We can merge tokens.txt and punctuations.txt files and make a new file including reading status and pronunciations. In this case we can read stand-alone characters. like: آ