Optimize state handling and passing
Closed this issue · 1 comments
Right now we are re-composing state a lot, or making database calls for the same state multiple times in parallel.
We could optimize our performance and exec time by caching state with a system that ensures that we always have the latest cache.
composeState queries actors, events, facts, conversation, etc. It will only be altered by the addition of new messages. Most functions have an optional state input, but some of these are not set up properly and make compose the state again anyways.
It's important that on elaborate and in the space between the user's message being saved and the agent responding that we update the conversation history and any other necessary state as needed locally while passing the state object through.
Implemented this, needed to speed up the Discord Bot and this was slowing us down a lot.