Official Llama 3.1 Evals

Question

YLGH opened this issue 5 months ago · 2 comments

Hey,

Is anyone currently working on adding the official Meta llama 3.1 eval dataset/tasks to helm?
https://huggingface.co/datasets/meta-llama/Meta-Llama-3.1-405B-Instruct-evals

Thanks!

Answer 1 · 2024-07-30T23:21:28.000Z

Thanks for the interest! We are working on the Llama 3.1 Instruct evaluations currently and will have them soon.

Answer 2 · 2024-08-08T17:14:42.000Z

Llama 3.1 Instruct Turbo evaluations have been released! (Lite, MMLU)