niieani/gpt-tokenizer
JavaScript BPE Tokenizer Encoder Decoder for OpenAI's GPT-2 / GPT-3 / GPT-4 / GPT-4o. Port of OpenAI's tiktoken with additional features.
TypeScriptMIT
Stargazers
- andrewexParse Pay
- antoineklleeSingapore
- cirolosapio@crewbe
- codsaneChicago, IL
- devinschulz@capeprivacy
- edwardfxiaoDunder Mifflin
- et84121Taiwan, Taipei
- faryhuo
- fengsky401
- fly51flyPRIS
- g-ongenae@algoan
- GreenRabite@latticehr
- HaustgeirrUK
- humanismusicNow
- InvertedX@Foundation-Devices
- iskandarrezaCalifornia
- jashionChina
- junedomingoRebel-Fitness
- kachar@team-gpt, @camplight
- leonardssh@liberty-mp • @bluestarplanning • @fluiter-gezocht
- llegomarkTeacherPH
- marcosgonzalezmarcialBarcelona, Spain
- mcnaveenThanjavur, India
- mikespragueCornell University
- MishaMgla
- mskianTamil Nadu, India
- nathanclevengerDriv.ly
- nwaughachukwumahbrulabs
- oieduardorabeloAuckland - NZ
- posvaFreelance
- rebzIndianapolis, IN
- samuelsurgeonSydney, Australia
- SandalotsVolcanak
- TenviLi@bilibili
- Zourunfashen zheng
- zpng