Slow tool calls compared to web
madroneropaulo opened this issue · 0 comments
Confirm this is an issue with the Python library and not an underlying OpenAI API
- This is an issue with the Python library
Describe the bug
I'm following the function call tutorial without streaming. So my code looks like this
# Add the user message to the thread
client.beta.threads.messages.create(
thread_id=thread_id,
role="user",
content=user_input,
)
print("thread created. Running run", datetime.now().time().strftime("%H:%M:%S"))
run = client.beta.threads.runs.create_and_poll(
thread_id=thread_id,
assistant_id=assistant_id,
)
if run.status == 'completed':
print("run complete", datetime.now().time().strftime("%H:%M:%S"))
messages = client.beta.threads.messages.list(
thread_id=thread_id
)
print(messages)
else:
print(run.status)
# Define the list to store tool outputs
tool_outputs = []
# Loop through each tool in the required action section
for tool in run.required_action.submit_tool_outputs.tool_calls:
print("tool received", datetime.now().time().strftime("%H:%M:%S"))
if tool.function.name == "get_weather":
...etc
It takes 4 seconds from the moment when the thread is created to the moment I get the function name and params
thread created. Running run 21:22:40
tool received 21:22:44
When I try the assistant from the assistants playground in the web dashboard, it takes 1500ms to return a message with the right function name. And I noticed the browser version makes an POST request to this endpoint which I guess is the same the python library uses under the hood:
https://api.openai.com/v1/threads/thread_id123/runs
I understand that that endpoint probably just starts the run, but still, the message with the function name loads in less than 2 seconds compared to 4. Am I using the right example for this? thanks
To Reproduce
Follow the function calling in this tutorial https://platform.openai.com/docs/assistants/tools/function-calling/quickstart?context=without-streaming and measure the time it takes from the moment the message is sent to the moment the function name is available in the handler. After that compare that result with the time it takes to perform the same action in the assistants playground in the web console.
Code snippets
No response
OS
Macos
Python version
3.9.6
Library version
latest