[Bug]: Tool_choice not supported with watsonx.ai.
Closed this issue · 3 comments
What happened?
In the updated watsonx.ai API, tool_choice and tools parameters are avaible but not mapped with liteLLM.
More documentation to use it here: https://cloud.ibm.com/apidocs/watsonx-ai#text-chat
Looks because the litellm only whitelists certain parameters for each LLM including watsonx (per what they support or not according to this page).
Relevant log output
No response
Twitter / LinkedIn details
No response
Interesting - looks like a new api from watsonx - /text/chat
this is a broader than expected ticket - but a good one.
It looks like the watsonx /text/chat endpoint is a lot more capable (supports messages, tool role, etc.)
I'm not sure if this is a beta endpoint or not, so we can begin by exposing it via a watsonx_chat/ route, and make it the default route if this ends up being the more stable option
Thank you for that!