Agent stopped due to max iterations
Opened this issue · 12 comments
Please disclose the principle of using agent in the Readme and provide the control of max iteration in.
It is still quite costly if run multiple iterations.
Hi @goldengrape , an autonomous agent is a concept from the LangChain library for GPT to take iterative actions based on user input and available information (context). More details can be found here: https://python.langchain.com/en/latest/modules/agents.html
Agent's ability to make incremental progress through iterative API calls is what gives it power to solve complex problems. However, sometimes it gets stalled after repeated tries until the max iterations.
In the short term, one potential way to alleviate this problem is through API caching, which is part of our road map.
Thank you!
I hope this could be resolved soon. Today, I had found the following token usage but I got no results from the query.
First query:
gpt-3.5-turbo-0301, 22 requests
28,762 prompt + 1,018 completion = 29,780 tokens
Second query:
gpt-3.5-turbo-0301, 15 requests
25,971 prompt + 919 completion = 26,890 tokens
Could this be related to the fact that there are too many items in the Zotero library? In my current Zotero library, there are more than 800 items. If this is the case, to partly address this issue, it would be great if one can query based on a certain sub-collection of a library. Maybe within this sub-collection, there are 10 items. Then the token usage would be much smaller?
Could this be related to the fact that there are too many items in the Zotero library?
Hi @wenlzhang , it is a valid hypothesis, although I can't be sure without knowing your questions. Would you mind sharing your questions so we can potentially use them for future optimization? Thanks
I can't be sure without knowing your questions
My questions and assumptions are as follows:
- My Zotero library contains too many items. This might require many tokens/iterations to handle, which may be one of the reasons for causing the current issue: "Agent stopped due to max iterations".
- Even if the current issue is not caused by this point, allowing to query based on a sub-collection may help reduce the token usage and make the answers more relevant.
Hi @wenlzhang thanks for the suggestions. Totally make sense to me.
Could you share the queries you used in Zotero RA, so I can test on my side?Thanks!
Could you share the queries you used in Zotero RA, so I can test on my side?
I just tested two queries and none of them worked for me. One of the queries is something like “How to design a controller to control a vehicle.”
I just tested two queries and none of them worked for me. One of the queries is something like “How to design a controller to control a vehicle.”
When I further reflect on this now, the max iterations issue could also be related to the fact that the items in my Zotero library cover may topics. Let's say, there are 800 items in total, but only 20 papers are closely related to my query. Still, the RA may need to handle all items and try to find related items. If not handled efficiently, this may easily cause issues, I assume.
@wenlzhang In the latest release, the chat window will display all the intermediate actions taken by the bot for each output. This hopefully gives you more insight into what it is doing and where it gets into problems.
@goldengrape The maximum iterations (steps) has been reduced from 15 to 6. Beyond that, it is most likely running in circles and won't produce meaningful output.
Please give it a try and let me know your feedback.
In the latest release, the chat window will display all the intermediate actions taken by the bot for each output. This hopefully gives you more insight into what it is doing and where it gets into problems.
Thank you for the update! Now I got the desired responses with version 0.0.8! I tested for both "Hello" and more specific questions as before.
Also, the token usage is much smaller now.
First query:
gpt-3.5-turbo-0301, 3 requests
2,945 prompt + 216 completion = 3,161 tokens
Second query:
gpt-3.5-turbo-0301, 3 requests
2,913 prompt + 609 completion = 3,522 tokens
Third query:
gpt-3.5-turbo-0301, 6 requests
6,667 prompt + 633 completion = 7,300 tokens
I brushed up against a similar issue. My question would be whether it would be possible the return at least "some" intermediate answer even when "max iteration stoppage" happens? If I understand correctly some information has been gained up to this point no?
(thanks for aria!)
Hi @andreifoldes The chatbot uses GPT to decide how to answer a question by breaking it up into several steps.
In your case, it 1) converted your question into a search query, 2) used the query to search in Zotero and then 3) tried to retrieve an article (item 194) for response synthesis. This process makes sense. But the last step somehow went wrong as the chatbot was repeatedly trying to fetch the same item - likely it was not able to get enough information to answer your question.
You can ask the chatbot to retrieve item 194 to take a look yourself.
I am currently working on a "routing" mechanism to give users more control over how a question should be answered. I invite you to give it a try when it is released.