/SharpToken

SharpToken is a C# library for tokenizing natural language text. It's based on the tiktoken Python library and designed to be fast and accurate.

Primary LanguageC#MIT LicenseMIT

Stargazers