DanielBalsam/surv_ai

hitting openAI rate limits

getorca opened this issue · 1 comments

getorca commented 
429, message='Too Many Requests', url=URL('https://api.openai.com/v1/chat/completions')
Traceback (most recent call last):
  File "[/mnt/Data/Projects/surv_ai/surv_ai/lib/llm/gpt.py](https://file+.vscode-resource.vscode-cdn.net/mnt/Data/Projects/surv_ai/surv_ai/lib/llm/gpt.py)", line 92, in _get_completion
    response.raise_for_status()
  File "[/Projects/surv_ai/venv/lib/python3.11/site-packages/aiohttp/client_reqrep.py](https://file+.vscode-resource.vscode-cdn.net/home/lawrence/Projects/surv_ai/venv/lib/python3.11/site-packages/aiohttp/client_reqrep.py)", line 1005, in raise_for_status
    raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 429, message='Too Many Requests', url=URL('https://api.openai.com/v1/chat/completions')
429, message='Too Many Requests', url=URL('https://api.openai.com/v1/chat/completions')
Traceback (most recent call last):
  File "[/mnt/Data/Projects/surv_ai/surv_ai/lib/llm/gpt.py](https://file+.vscode-resource.vscode-cdn.net/mnt/Data/Projects/surv_ai/surv_ai/lib/llm/gpt.py)", line 92, in _get_completion
    response.raise_for_status()
  File "[/Projects/surv_ai/venv/lib/python3.11/site-packages/aiohttp/client_reqrep.py](https://file+.vscode-resource.vscode-cdn.net/home/lawrence/Projects/surv_ai/venv/lib/python3.11/site-packages/aiohttp/client_reqrep.py)", line 1005, in raise_for_status
    raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 429, message='Too Many Requests', url=URL('https://api.openai.com/v1/chat/completions')

Wondering how you got around this, trying to run some of the examples...

DanielBalsam commented

Hello! The LLM clients automatically implement a back off - so a few rate limits is not necessarily an issue! The survey response contains the number of errors as well so you can inspect how many agents truly failed.

However if the back off is not sufficient, there is also a parameter called "max_concurrency" available on both the Survey and tool classes. You can set this number lower than the default of 10 and that should help!

  • 👍1