openai/openai-python

Slow tool calls compared to web

madroneropaulo opened this issue · 0 comments

Confirm this is an issue with the Python library and not an underlying OpenAI API

  • This is an issue with the Python library

Describe the bug

I'm following the function call tutorial without streaming. So my code looks like this

 # Add the user message to the thread
    client.beta.threads.messages.create(
        thread_id=thread_id,
        role="user",
        content=user_input,
    )
    print("thread created. Running run", datetime.now().time().strftime("%H:%M:%S"))

    run = client.beta.threads.runs.create_and_poll(
        thread_id=thread_id,
        assistant_id=assistant_id,
    )
    
    if run.status == 'completed':
        print("run complete",  datetime.now().time().strftime("%H:%M:%S"))
        messages = client.beta.threads.messages.list(
        thread_id=thread_id
    )
        print(messages)
    else:
        print(run.status)
    
    # Define the list to store tool outputs
    tool_outputs = []
    
    # Loop through each tool in the required action section
    for tool in run.required_action.submit_tool_outputs.tool_calls:
        print("tool received",  datetime.now().time().strftime("%H:%M:%S"))
        if tool.function.name == "get_weather":
        ...etc
        

It takes 4 seconds from the moment when the thread is created to the moment I get the function name and params

thread created. Running run 21:22:40
tool received 21:22:44

When I try the assistant from the assistants playground in the web dashboard, it takes 1500ms to return a message with the right function name. And I noticed the browser version makes an POST request to this endpoint which I guess is the same the python library uses under the hood:
https://api.openai.com/v1/threads/thread_id123/runs
I understand that that endpoint probably just starts the run, but still, the message with the function name loads in less than 2 seconds compared to 4. Am I using the right example for this? thanks

To Reproduce

Follow the function calling in this tutorial https://platform.openai.com/docs/assistants/tools/function-calling/quickstart?context=without-streaming and measure the time it takes from the moment the message is sent to the moment the function name is available in the handler. After that compare that result with the time it takes to perform the same action in the assistants playground in the web console.

Code snippets

No response

OS

Macos

Python version

3.9.6

Library version

latest