[New Feature]: expose callbacks and ainvoke
Closed this issue · 6 comments
Checked for duplicates
- Yes
Alternatives considered
- Yes - and alternatives don't suffice
Related problems
- No, see below
Describe the feature request
However, in order for the GUI to stay responsive, deferring long-running tasks is essential. At the moment it is not possible to use ROSA for intermediate updates or react to errors. It would be nice if the ROSA constructor accepted a callbacks parameter to pass to its AgentExecutor. Exposing the ainvoke method would be useful, too.
As an aside, I'm not sure if adding double underscores to all class members is really necessary. It makes inheritance and monkey patching much more annoying :)
Can you elaborate on the intended use case a bit more? I'd like to understand how the callbacks would be used to keep the UI responsive.
Is it that you want to be able to submit arbitrarily many queries to the agent without waiting for the previous query to finish?
How will you keep track of the queries, their intermediate steps, and results?
Mostly just curious, I think it's a good idea either way.
Sure! Right now I'm running the request inside a thread and just update the UI from there once it returns. There are a couple of things I can't do with this approach (at least as far as I can tell):
- stream responses
- give user feedback about what tools are used
- react to failed tool runs
- cancel response generation
From what I saw, langchain uses a callback class and its member functions for handling response updates. I would construct this as I pass the request, then use a closure on my request and generated UI elements to keep the association.
Check out the astream implementation in rosa.py on feature/streaming. It's using LangChain astream_events method, which provides access to both streaming tokens, tool usage, etc.
Also see turtle_agent demo script for example usage.
One thing I noticed, and have yet to find a solution to, is that LangChain AgentExecutor cannot be pre-empted or cancelled with standard LangChain facilities. This will likely require use of asyncio tasks. What is unclear to me at the moment is whether ROSA should provide that integration, or if it should be left to the consumer (I am leaning more toward the latter).
Thank you! I'll have some time next week to test it out. I agree though that supporting request cancellation sounds like something langchain should provide.
Tried it out just now and works like a charm! The example was already very helpful. Seems like you had some fun with it :D Thank you!