sashabaranov/go-openai

index out of bound when streaming with IncludeUsage as true

Opened this issue · 1 comments

To Reproduce
When streamed chat completion with request having StreamOptions defined as -

req := openai.ChatCompletionRequest{
	Model:     openai.GPT3Dot5Turbo,
	Messages: []openai.ChatCompletionMessage{
		{
			Role:    openai.ChatMessageRoleUser,
			Content: "Lorem ipsum",
		},
	},
	StreamOptions: &openai.StreamOptions{IncludeUsage: true},
	Stream:        true,
}

breaks for last chunk-

panic: runtime error: index out of range [0] with length 0

goroutine 1 [running]:
main.main()
	/Users/aryan/Desktop/go-openai/examples/completion/main.go:60 +0x4b0
exit status 2

Reason for it - https://community.openai.com/t/usage-stats-now-available-when-using-streaming-with-the-chat-completions-api-or-completions-api/738156

Expected behavior
The condition to his the EOF error and stop the streaming process.

Environment (please complete the following information):

  • go-openai version: master
  • Go version: go version go1.22.3 darwin/arm64
  • OpenAI API version: v1

For anyone facing issue till this is fixed, can rely on FinishReason explicitly and return if present
or check the usageOptions and return if the usage chunk is present.

response, err := stream.Recv()
hasUsageTokens := response.Usage

if errors.Is(err, io.EOF) || hasUsageTokens != nil {
	return
}