word count limit
KrisFlowerss opened this issue · 4 comments
After deployment, I found that there is a limit to the length of the answers. It automatically stops when reaching the maximum limit. How can I change this limit?
I guessing it's the limitation caused by maxOutputTokens See here
@KrisFlowerss Actually, it is this parameter that affects the length of the output text.
If you're deploying on Docker, you can define an environment variable for it as much as I did, or if you don't need to use an environment variable, then you can just pin the value of <=30720 directly to this parameter (why <=30720? Because I noticed that in Google AI Studio, the maximum value of the Chat prompt is 30720)
Changes need to be made in route.ts or you can see my fork route.ts
I guessing it's the limitation caused by maxOutputTokens See here
@KrisFlowerss Actually, it is this parameter that affects the length of the output text. If you're deploying on Docker, you can define an environment variable for it as much as I did, or if you don't need to use an environment variable, then you can just pin the value of <=30720 directly to this parameter (why <=30720? Because I noticed that in Google AI Studio, the maximum value of the Chat prompt is 30720) Changes need to be made in route.ts or you can see my fork route.ts
you're right . I change numebr to 30000, it worked. thanks
Do you want me to add a feature in the frontend to limit the numbers of tokens send to the frontend ?