niieani/gpt-tokenizer

How to set the model in encode / encode generator like encodeChat or encodeChatGenerator?

Closed this issue ยท 1 comments

How to set the model in encode / encodeGenerator like encodeChat or encodeChatGenerator?

import { encode } from 'gpt-tokenizer'

const text = 'Hello, world!'
const generator =encodeGenerator(text, /* I want to add model or decoder here */)

like

import { encodeChatGenerator } from 'gpt-tokenizer'

const generator = encodeGenerator(messages, 'gpt-3.5-turbo')

๐ŸŽ‰ This issue has been resolved in version 2.2.1 ๐ŸŽ‰

The release is available on:

Your semantic-release bot ๐Ÿ“ฆ๐Ÿš€