Streaming Fails Due to `{"include_usage": True}`
sixwaaaay opened this issue · 1 comments
sixwaaaay commented
Description
When using Mistral AI's OpenAI-compatible API with streaming output Runner.run_streamed, the hardcoded stream_options={"include_usage": True} in the code triggers a 422 Unprocessable Entity error. Mistral's API does not support the include_usage parameter in stream_options, causing compatibility issues.
Root Cause:
Proposed Fix
Add an override mechanism for stream_options
Error
openai.UnprocessableEntityError: Error code: 422 - {'object': 'error', 'message': {'detail': [{'type': 'extra_forbidden', 'loc': ['body', 'stream_options', 'include_usage'], 'msg': 'Extra inputs are not permitted', 'input': True}]}, 'type': 'invalid_request_error', 'param': None, 'code': None}