how to evaluate gemini-1.5 pro

Question

how to evaluate gemini-1.5 pro

Closed this issue 2 months ago · 2 comments

Thank you so much for providing such a wonderful job. I am trying to reproduce the evaluation result of Gemini-1.5 pro. However I can not find any script in the repo. Would you mind to share the script for reference or give me some hint of how to evaluate.

Answer 1 · 2024-07-22T06:30:56.000Z

Hi roy, thanks for your attention. You could study how to use gemini, and if you have any questions, please let me know.

Answer 2 · 2024-07-22T07:13:33.000Z

Thanks for you reply. I have also found some wonderful reference for evaluating models on video-mme. For anyone have interest, here is some code might useful for evaluating Video-MME. https://github.com/EvolvingLMMs-Lab/lmms-eval. Hope that helps. Thanks again.