/Tokenizer

Some demo tokenizers especially for Chinese, including Maximum Matching, UniGram, HMM, CRF.

Primary LanguagePython

No issues in this repository yet.