[Bug]: `cohere-embed-v3` models fail via litellm proxy while they work fine directly via Azure AI

[taralika]

What happened?

Make the following request via proxy:

curl https://proxy-host/embeddings \
  -H 'Content-Type: application/json' \
  -H "api-key: $api_key" \
  -d '{"model":"cohere-embed-v3-english","input":["Dogs","Cats"]}'

response:

{
	"error": {
		"message": "litellm.BadRequestError: Azure_aiException - {\"error\":{\"code\":\"Invalid input\",\"message\":\"{\\\"message\\\":\\\"invalid data uri\\\"}\",\"status\":422}}\nReceived Model Group=cohere-embed-v3-english\nAvailable Model Group Fallbacks=None",
		"type": null,
		"param": null,
		"code": "400"
	}
}

Now, make the same request directly to the same model in Azure:

curl https://cohere-embed-v3-english-8834he8u0dskhj3248.westus.models.ai.azure.com/embeddings \
  -H "Authorization: Bearer $azure_token" \
  -H "Content-Type: application/json" \
  -d '{"input":["Dogs","Cats"]}'

works fine, response:

{
	"id": "324a34-6e31-43ec-a2de-e393b96faab4",
	"object": "list",
	"data": [
		{
			"index": 0,
			"object": "embedding",
			"embedding": [
				-0.0022850037,
				0.04220581,
				...
			]
		},
		{
			"index": 1,
			"object": "embedding",
			"embedding": [
				-0.0025520325,
				0.024780273,
				...
			]
		}
	],
	"model": "embed-english-v3.0",
	"usage": {
		"prompt_tokens": 2,
		"completion_tokens": 0,
		"total_tokens": 2
	}
}

Noticed the issue with both cohere-embed-v3-english and cohere-embed-v3-multilingual models. The issue doesn't happen always, but it does happen for specific inputs like the one given in above example.

E.g., a request like this works fine via the proxy:

curl https://proxy-host/embeddings \
  -H 'Content-Type: application/json' \
  -H "api-key: $api_key" \
  -d '{"model":"cohere-embed-v3-english","input":["Dogs are my friends","Cats are my friends"]}'

But again, the following request fails:

curl https://proxy-host/embeddings \
  -H 'Content-Type: application/json' \
  -H "api-key: $api_key" \
  -d '{"model":"cohere-embed-v3-english","input":["Dogsaremyfriends", "Catsaremyfriends"]}'

Relevant log output

No response

Twitter / LinkedIn details

No response

[krrishdholakia]

able to repro