Support Chinese
thisandagain opened this issue · 0 comments
thisandagain commented
Add support for simplified and traditional Chinese. This will require a modification to the strategy for tokenization of the input phrase.
References
https://nlp.stanford.edu/software/segmenter.shtml
https://github.com/yishn/chinese-tokenizer