szegedai/byte-offset-tokenizer

Tokenizer for our byte based transformer model. See: https://huggingface.co/SzegedAI/charmen-electra

Python