Frequent Error 500 when working with images in v0.4.8 & 0.4.4.

[anupdsouza]

Description of the bug:

Since yesterday, I'm seeing a lot of 500 error code responses when making requests. Here is a request I made with the latest version of the GenerativeAIMultimodalSampleApp using one of the stock photo library images with a valid API key (removed from the request for privacy).

curl -H 'x-goog-api-key: ' -H 'Content-Type: application/json' -H 'x-goog-api-client: genai-swift/0.4.8' 'https://generativelanguage.googleapis.com/v1/models/gemini-1.0-pro-vision-latest:streamGenerateContent?alt=sse' -d '{"contents":[{"role":"user","parts":[{"text":"Look at the image(s), and then answer the following question: Is this a real waterfall ?"},{"inline_data":{"data":"\/9j\/4AAQSkZJRgABA...
------------------------
[GoogleGenerativeAI] The server responded with an error: <NSHTTPURLResponse: 0x600000350a20> { URL: https://generativelanguage.googleapis.com/v1/models/gemini-1.0-pro-vision-latest:streamGenerateContent?alt=sse } { Status Code: 500, Headers {
    "Alt-Svc" =     (
        "h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000"
    );
    "Content-Length" =     (
        200
    );
    "Content-Type" =     (
        "text/event-stream"
    );
    Date =     (
        "Mon, 18 Mar 2024 05:54:51 GMT"
    );
    Server =     (
        "scaffolding on HTTPServer2"
    );
    Vary =     (
        Origin,
        "X-Origin",
        Referer
    );
    "server-timing" =     (
        "gfet4t7; dur=4131"
    );
    "x-content-type-options" =     (
        nosniff
    );
    "x-frame-options" =     (
        SAMEORIGIN
    );
    "x-xss-protection" =     (
        0
    );
} }
[GoogleGenerativeAI] Response payload: {
  "error": {
    "code": 500,
    "message": "An internal error has occurred. Please retry or report in https://developers.generativeai.google/guide/troubleshooting",
    "status": "INTERNAL"
  }
}

Streaming also seems broken when used with the older v0.4.4 of the SDK as can be seen in my own project. I'm also experiencing a lot of error 500 responses. The request & response is as follows:

[GoogleGenerativeAI] Creating request with the equivalent cURL command:
----- cURL command -----
curl -H 'x-goog-api-key: ' -H 'Content-Type: application/json' -H 'x-goog-api-client: genai-swift/0.4.4' 'https://generativelanguage.googleapis.com/v1/models/gemini-1.0-pro-vision-latest:streamGenerateContent?alt=sse' -d '{"contents":[{"role":"user","parts":[{"text":"Is this a real waterfall ?"},{"inline_data":{"data":"\/9j\/4AAQSkZJRgABAQAASABIAA...
------------------------
[GoogleGenerativeAI] Stream response: data: {"candidates": [{"content": {"parts": [{"text": " Yes, this is a real waterfall. It is called Skogafoss, and it is located in Iceland. The waterfall is 60 meters high and"}],"role": "model"},"finishReason": "STOP","index": 0,"safetyRatings": [{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}]}],"promptFeedback": {"safetyRatings": [{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}]}}

This doesn't seem like a one off network issue as I've tried to make requests at different times of the day but image related requests are failing way too often. I had to truncate the image data due to post character limits.

Actual vs expected behavior:

Given a valid api key, the sdk should return valid responses.

Any other information you'd like to share?

No response

[paulb777]

@anupdsouza Sorry about the trouble. We had several service related issues last week that caused the 500 errors.

Please retry and let us know if it continues to be a problem.

If you're seeing issues with only 0.4.4, please update to the latest version, currently 0.4.8.

[anupdsouza]

Hi @paulb777 , I can confirm that the issues are resolved now on both 0.4.4 & 0.4.8 versions of the SDK.