/shami-token

Shami Language Model Tokenizer

Primary LanguagePython

license
apache-2.0

Shami Tokenizer

This is the tokenizer of Shami Model.