/Japanese-LUW-Tokenizer

Japanese Long-Unit-Word Tokenizer with RemBertTokenizerFast of Transformers

Primary LanguagePythonMIT LicenseMIT

No issues in this repository yet.