Azure/azure-sdk-for-go

Add support for usage stats when streaming chat completion

Closed this issue · 2 comments

ejiangshan commented

Feature Request

Do we have any plan to support this feature as stated in #22940?

github-actions commented

Thank you for your feedback. Tagging and routing to the team member best able to assist.

richardpark-msft commented

As of the latest package you can set IncludeUsage in the StreamOptions field:

https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/ai/azopenai#ChatCompletionsStreamOptions

body.StreamOptions = &azopenai.ChatCompletionStreamOptions{
    IncludeUsage: to.Ptr(true),
}