[BUG] Gemini Model fails to invoke any tools.
essamamdani opened this issue · 8 comments
Is there an existing issue for this?
- I have searched the existing issues
Vercel Runtime Logs
- I have checked the Vercel Runtime Logs for errors (if applicable)
Current Behavior
I switched from OpenAI to Google Gemini Flash 1.5. It now directly provides answers using the LLM. However, after two hours, I noticed the 'toolChoice: required' setting in the Vercel AI SDK. As a result, the output continuously triggered search cards without stopping the process or generating responses based on the search results.
Here is some snaps:
Expected Behavior
It should function the same way as OpenAI’s GPT-4 version.
Steps To Reproduce
- I just added the Generative API key to the
.envfile. - After that, I received answers directly from the LLM without any tools being executed.
- Later, I found the
toolChoice: 'required'option and added it to theresearcher.tsxfile.
Environment
- OS:Mac
- Browser:ChromeAnything else?
"@ai-sdk/google": "0.0.23",
"@google/generative-ai": "^0.3.1",
"ai": "^3.2.10",
This error is an issue with the Gemini model. If toolChoice is set to auto (default), the model decides whether to call the tool. If toolChoice is set to required, the model will call the tool as needed. This also depends on the model.
In the latest code, maxSteps is set to 5, so the tool should not continue to be called: https://github.com/miurla/morphic/blob/main/lib/agents/researcher.tsx#L30
https://sdk.vercel.ai/docs/ai-sdk-core/tools-and-tool-calling#multi-step-calls
Please try with the latest code.
https://github.com/miurla/morphic/blob/main/package.json#L15
it's still the same error. After I updated the packages
Still Don't call any tools:
it's still the same error. After I updated the packages
No, I explained that in the latest version, the issue of continuously calling tools has been resolved.
I found that there are limitations in the schema of the Google Generative API: https://sdk.vercel.ai/providers/ai-sdk-providers/google-generative-ai#troubleshooting-schema-limitations
PR: #358
It started calling tools. The model I confirmed was gemini-1.5-pro-002. gemini-1.5-flash-002 did not call tools. This should be dependent on the model.
Thanks Let me check.
Followed these steps:
git clone https://github.com/miurla/morphic
cd morphic
bun install
nano .env added GOOGLE_GENERATIVE_AI_API_KEY and other necessary environment variables
bun run dev
��
Did you tried some query? The model decides whether to call the tool
How can we make forceable to call tool everytime before generating the answer?