Already subscribed but still having an error message
Opened this issue · 1 comments
Katniss27 commented
spammenotinoz commented
@mckaywrigley , I reduced my API usage costs by 98% through two key actions;
a) Quick code change to limit conversation history
And
b) Rephraser PULL request from @fkesheh #1518
For a) Some users just use a single conversation that is months old, this means for every single message, even as simple as "what model is this?", the entire conversation is sent back and forth in each API query which can be many thousands of tokens. Costs a friggin bomb. Found this by using CloudFlares AI gateway.
Given your user volume, you may also benefit from caching, refer #1527