Azure/azure-sdk-for-net

Setting Response Format to JSON for OpenAI Assistant

Opened this issue · 5 comments

raysuelzer commented

Library name and version

Azure.AI.OpenAI.Assistants_1.0.0-beta.3

Query/Question

I cannot seem to find anywhere in the Assistants SDK to set the "response format" when using the OpenAI SDKs. This is useful to ensure that the API returns valid JSON. Sorry if this exists but I am missing it.

From OpenAI:

https://platform.openai.com/docs/api-reference/chat/create#chat-create-response_format

response_format
object

Optional
An object specifying the format that the model must output. Compatible with [GPT-4 Turbo](https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo) and all GPT-3.5 Turbo models newer than gpt-3.5-turbo-1106.

Setting to { "type": "json_object" } enables JSON mode, which guarantees the message the model generates is valid JSON.

Important: when using JSON mode, you must also instruct the model to produce JSON yourself via a system or user message. Without this, the model may generate an unending stream of whitespace until the generation reaches the token limit, resulting in a long-running and seemingly "stuck" request. Also note that the message content may be partially cut off if finish_reason="length", which indicates the generation exceeded max_tokens or the conversation exceeded the max context length.

Environment

No response

github-actions commented

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jpalvarezl @trrwilson.

raysuelzer commented

Any help on this would be appreciated. I'm having some real issues gettng the assistant to return JSON responses.

nighby commented

@raysuelzer as a temporary workaround, you can visit https://platform.openai.com/assistants using your account (the one tied to the API key you are using to create assistants), click on your assistant and toggle 'JSON object' under Response format

medoni commented

Still missing in 1.0.0-beta.4

TomasHubelbauer commented

Ideally, this would also support the json_schema value introduced with structured outputs, not just json_object:
https://openai.com/index/introducing-structured-outputs-in-the-api/.

FWIW as of 2.0.0-beta2, neither value is supported.

  • 👍3