huggingface/tokenizers
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
RustApache-2.0
Stargazers
- amilajack@palettedev
- billykwok@amzn
- cfoster0
- deepnarainsinghGalvanizeU
- dnanhkhoa@aistairc
- dorringel
- EntilZha@facebookresearch, FAIR Labs
- fly51flyPRIS
- galsangHanyang University
- googlx
- h3lio5
- icochico@ServiceNow
- imbibekkSeoul
- jiangtianli91
- jonbuffingtonLake Forest, IL, USA
- jparr721@EspressoSystems
- kalyangvsUniversity of Florida
- kaushaltrivedi
- labardan
- lamptsSaigonapps
- leslyarun
- LuvataHanoi - Vietnam
- MarkusOdenthalCologne, Germany
- mmarius
- mrluanmaShanghai, China
- partmorTurin
- popeye007
- pushpendrapratapNew Delhi, India
- ranpoxC-137
- rish-16@NUSSOC @caraml-group
- ryuseonghanSamsung Research
- Shujian2015Google
- starlureSeattle
- tekumaraTerminate and stay resident
- xanhho
- zjm008Penang