skeskinen/bert.cpp

How can I increase n_max_tokens?

Opened this issue · 1 comments

ztkg257 commented

After modifying the value of n_max_tokens in bert.cpp from "int32_t n_max_tokens = 512;" to "int32_t n_max_tokens = 10000;", I proceeded to rebuild the project. However, upon testing, the value of n_max_tokens remained unchanged at 512 despite the modification.
image

skeskinen commented

max tokens is a property of the loaded model. Different models might have different values.