javirandor/anthropic-tokenizer
Approximation of the Claude 3 tokenizer by inspecting generation stream
PythonMIT
Stargazers
- andrewflynn20
- brilandSRI International @SRI-CSL
- cern1710United Kingdom
- chuanmingliuWesteros
- creo13
- denisfitz57
- dhruv-anand-aintechAI Northstar Tech
- diegoasua
- dpaleka
- Eris2025
- fpargaMontreal, QC
- gaozhitongEPFL
- gm8xx8
- jaanliUniversity of Tartu & @onefact
- jio-gl
- JohnHeibelEugene, Oregon
- karanvora2599NYU Langone Health
- kushalthaman@physoly, Online Physics Olympiad
- LericDaxAzoth Corp
- lightmatmul@Dynamofl
- llegomarkTeacherPH
- matrix-compute
- menegazzi
- mikeoller82
- Minami-su
- ndronenThe Great American Desert
- ocriado91
- opooladzLos Angeles
- oulianovphospho
- pszemrajAtlanta, GA
- s4mw1se
- simonwDatasette
- Taylorwk
- temberatureUSA
- tkersey@thisisartium
- vaibhavk97New Delhi , India