huggingface/tokenizers
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
RustApache-2.0
Watchers
- alfellati@openweb3-foundation | @setheum-js | @Setheum-Labs | @Slixon-Technologies
- ArunkumarRamananAI Founder @Deep-Brainz & Stealth AI Labs
- beurkinger@huggingface
- bmwas
- BritneyMuller@datasci101
- chenjianxinTaipei
- codifierrIndia
- coolshinejonson
- CristaLCorp
- DanielHesslowAdaptive
- DN6@huggingface
- douwekiela
- drbhdrbh
- duyvuleo@oracle
- dylanebertHugging Face
- eunseojoPalo Alto
- evijit@huggingface
- fajri91MBZUAI, IndoNLP
- graniet00 00 80 00
- HavorldPeking
- heleifzAlibaba
- hydra-zim
- juliensimonHugging Face
- liuhangyubeijing
- mariusz-jachimowicz-83Poland
- meganrileyChicago
- mhetrerajatSeattle, WA
- mishig25@huggingface
- nazneenrajani@huggingface
- omarespejelStarkware
- oOraph
- QilinGuPortland, Oregon
- rdgozumPhilippines
- rigpalab
- rwightmanVancouver, BC
- yjerniteCIMS, NYU