hkust-nlp/AgentBoard

[Request] Google Gemini Pro

abdinal1 opened this issue · 2 comments

abdinal1 commented

Have you or can you evaluate Google Gemini Pro to put as comparison?

chang-github-00 commented

Thanks for the suggestion, though we have not yet tested the performance of Google Gemini Pro on AgentBoard. We would evaluate it once we have the resources.
BTW, we also welcome everyone to contribute evaluation results of other models to our email. If you have run Google Gemini Pro on AgentBoard, you can kindly send the log file to our email (llmagentboard@gmail.com).

chang-github-00 commented

We've recently add evaluation on Google Gemini-flash, please check our updated arxiv.