/llama3-tokenizer-js

JS tokenizer for LLaMA 3 and LLaMA 3.1

Primary LanguageJavaScriptMIT LicenseMIT

Stargazers