Pure Go implementation of OpenAI's tiktoken tokenizer
Primary LanguageGoMIT LicenseMIT
No one’s watching this repository yet.