dqbd/tiktoken

Issues

• Expose getEncodingNameForModel in wasm version?

  #123 opened a month ago by jens-f
  0

• recursive use of an object detected which would lead to unsafe aliasing in rust

  #119 opened 4 months ago by mogery
  2

• React native error SyntaxError: Invalid RegExp: Invalid escape] in react native expo.

  #110 opened 5 months ago by djaffer
  1

• It doesn't support new model "o1-mini" and "o1-preview"

  #120 opened 3 months ago by Talented-Business
  8

• WASM file is not included in Next.js Node runtime API routes

  #67 opened 3 months ago by AaronFriel
  0

• no such file or directory,_tiktoken_bg.wasm

  #60 opened 3 months ago by kiraraty
  1

• Cloudflare worker Uncaught ReferenceError: FinalizationRegistry is not defined

  #46 opened a year ago by EiffelFly
  8

• Document different package options

  #48 opened 3 months ago by transitive-bullshit
  2

• Error: null pointer passed to rust

  #69 opened a year ago by emeeks
  4

• NPM package is very huge

  #68 opened a year ago by anzemur
  6

• missing example code for js-tiktoken

  #102 opened 8 months ago by coding22
  1

• [BUG] library works in test, but fails in dev server

  #55 opened 4 months ago by dhruv-anand-aintech
  1

• NPMJS repository versions are out of date

  #63 opened 4 months ago by kjvenalainen
  2

• An error occurred during the deployment of the Vercel edge function

  #93 opened 4 months ago by bowencool
  2

• Missing ranks files from js-tiktoken

  #108 opened 4 months ago by jonasb
  2

• Add `gpt-4o-mini` model

  #111 opened 4 months ago by huytool157
  2

• How to support Llama 3?

  #100 opened 4 months ago by wizd
  1

• missing TiktokenModel for version 3.5-turbo-16k

  #74 opened 4 months ago by hannahlaek-wt
  2

• gpt-3.5-turbo-instruct support

  #73 opened 4 months ago by danny-avila
  2

• gpt-4o-mini Error: Unknown model at getEncodingNameForModel

  #114 opened 4 months ago by 0xinhua
  1

• Add gpt-4o-2024-08-06 Model

  #115 opened 4 months ago by itarutomy97
  1

• Using Tiktoken WASM in React Environment

  #113 opened 5 months ago by djaffer
  0

• Add support for the newest Open AI embeddings models

  #90 opened a year ago by AleksTrpkovski
  2

• Add `gpt-4o` support

  #105 opened 7 months ago by huytool157
  1

• Add `gpt-4.5-turbo` support

  #95 opened 7 months ago by huytool157
  0

• Performance

  #99 opened 8 months ago by steebchen
  0

• Add `gpt-4-turbo` support

  #96 opened 8 months ago by huytool157
  0

• Cannot find module error

  #75 opened a year ago by thdoan
  1

• Add decode_with_offsets function (or PR guidance request)

  #94 opened 9 months ago by Sam1320
  0

• Memory access out of bounds

  #84 opened a year ago by maxdeichmann
  3

• Cloudflare worker Uncaught ReferenceError: FinalizationRegistry is not defined

  #92 opened 10 months ago by TranquilMarmot
  1

• 'TextEncoder' is not defined - MongoDB Functions

  #91 opened 10 months ago by georgeherby
  1

• Support Latest Models

  #83 opened a year ago by johannbuscail
  1

• Add `gpt-4-turbo-preview` to `TiktokenModel`

  #85 opened a year ago by huytool157
  3

• Crash on React Native: `SyntaxError: Invalid RegExp: Invalid escape`

  #82 opened a year ago by julian-hecker
  2

• Plans for gpt-4-turbo variants?

  #80 opened a year ago by ianschmitz
  3

• How to count tokens for functions defined in the prompt?

  #78 opened a year ago by vrde
  1

• Is it reasonable to load and run on the client side ?

  #77 opened a year ago by MarArMar
  1

• Support fine tunned models

  #76 opened a year ago by italojs
  1

• What does free actually do?

  #72 opened a year ago by disbelief
  9

• Raise error when concurrent to call new Tiktoken

  #70 opened a year ago by qkxie
  2

• Plans for `gpt-3.5-turbo-16k`?

  #59 opened a year ago by khill-fbmc
  5

• When running 1.0.7 there is a Missing tiktoken_bg.wasm error on AWS Lambda typescript

  #61 opened a year ago by haibh-wanosoft
  3

• js-tiktoken ES5 support

  #64 opened a year ago by haolong7
  2

• When running 1.0.9 there is a Missing tiktoken_bg.wasm error

  #56 opened a year ago by jensen
  1

• Does this work in browser ?

  #47 opened a year ago by snwfdhmp
  2

• Fatal crash: BasicBlock requested for unrecognized address

  #62 opened a year ago by SamStenner
  1

• When I look at cl100k_base.json, I don't see the special tokens like <|im_start|>, but when I add them manually I get an error from the wasm

  #65 opened a year ago by Evertt
  2

• [js-tiktoken] memory leak

  #50 opened 2 years ago by louis030195
  3

• I would like to ask you to help me integrate into cloudflare worker for a fee, is anyone willing to help me?

  #49 opened 2 years ago by SunnyGPT
  0