niieani/gpt-tokenizer
JavaScript BPE Tokenizer Encoder Decoder for OpenAI's GPT-2 / GPT-3 / GPT-4. Port of OpenAI's tiktoken with additional features.
TypeScriptMIT
Stargazers
- 4ortytwoAnlatan
- BedirTFathomAI
- cantrell@adobe
- ccamel@axone-protocol
- chuanqisun@Microsoft
- corleone113
- coryasilvaVista, California
- cvic@code4romania
- demironmanx
- dzhy-li
- eamodio@gitkraken
- earlAchromatic
- famasoonTokyo, Japn
- ForeverMEhaha
- haifengkao
- JimmyLv@ParabolInc
- JustGregVienna, Austria
- kafkas@ProficientAI
- KittenYangAlibaba.Inc
- marcusrbrown
- Max-MakhrovSheets with Max
- morganhvidt@puri-io
- nikitavoloboevTbilisi
- patxbert
- rpPandaWorkduck
- RyanLenChina
- ryderstormGreensboro, NC
- sbusso@kyrolabs
- SchlagerKhanSweden
- sinedied@Microsoft
- sixersKrakow
- ThomasGHenryThomas G Henry LLC
- tthallosChangsha
- worstpracticeSweden
- yassine-cc
- zenoamaro@matterway