Azure/azure-rest-api-specs

[Question] When Azure will sync OpenAI ChatCompletion stream_options feature ?

SKYhuangjing opened this issue · 25 comments

SKYhuangjing commented

API Spec link

https://github.com/Azure/azure-rest-api-specs/blob/main/specification/cognitiveservices/data-plane/AzureOpenAI/inference/preview/2024-04-01-preview/inference.json

API Spec version

2024-04-01-preview

Question/Query

OpenAI ChatCompletion API has add stream_options {include_usage} for stream get usage, it will help developer to statistics the api cost, it's very good for us, so when azure will sync it?

Environment

REST API invoke it

zzhxiaofeng commented

@YunsongB Please help have a look, thank you.

supuwoerc commented

+1+1+1

janaka commented

I'm guessing this all now comes along with the new OpenAI client that pulled in v2.0.x

dq7532183 commented

when?

sandangel commented

Hi, I tried with 2024-05-01-preview but still get the error. 400 bad request. I think it needs to be enabled on the server side, not client.