vertexai.gemini does not respect timeout
Closed this issue · 1 comments
exaV commented
I get the following timeout exception when invoking gemini:
jakarta.ws.rs.ProcessingException: The timeout period of 10000ms has been exceeded while executing POST /v1/projects/myrealproject/locations/us-west1/publishers/google/models/gemini-1.5-flash:generateContent for server null
I tried the following configuration but it seems to have no effect:
quarkus.langchain4j.timeout=30s
I also tried adding a org.eclipse.microprofile.faulttolerance.Timeout
annotation to my @RegisterAiService
but that has no effect either.
How do I configure the timeout for my an AiService backed by gemini? I could not find any properties in devui or in the documentation.
I found this bit of code in the implementation, which to me suggests that it is simply not configurable yet and does not inherit the default either, but I might be wrong.
- https://github.com/quarkiverse/quarkus-langchain4j/blob/main/model-providers/vertex-ai-gemini/runtime/src/main/java/io/quarkiverse/langchain4j/vertexai/runtime/gemini/VertexAiGeminiRecorder.java#L52
- https://github.com/quarkiverse/quarkus-langchain4j/blob/main/model-providers/vertex-ai-gemini/runtime/src/main/java/io/quarkiverse/langchain4j/vertexai/runtime/gemini/VertexAiGeminiChatLanguageModel.java#L87
geoand commented
That indeed looks like the case. Would you like to provide a fix for this?
Thanks