dqbd/tiktoken

Fatal crash: BasicBlock requested for unrecognized address

SamStenner opened this issue · 1 comments

Hi, we're occasionally encountering the following error in our NodeJS application:

Estimating tokens for text-davinci-003
Getting encoding for text-davinci-003
assertion failed [block != nullptr]: BasicBlock requested for unrecognized address
(BuilderBase.h:550 block_for_offset)
[1]    89655 trace trap  ts-node index.ts

Version: 1.0.7

Seems to be some sort of web assembly issue.

dqbd commented

Hello @SamStenner, sorry for the delay!
Could it be possible that the encoder has not been free'd after prolonged usage? Would it be possible to share a code example?