/TiktokenSharp

Token calculation for OpenAI models, using `o200k_base` `cl100k_base` `p50k_base` encoding.

Primary LanguageC#MIT LicenseMIT

Stargazers