/TokenizationBenchmarks

Comparison of various supervised and unsupervised tokenization algorithms on a Chinese corpus

Primary LanguagePython

Stargazers

No one’s star this repository yet.