TIGER-AI-Lab/MMLU-Pro

Which DeepSeek-Coder-V2?

Closed this issue · 1 comments

The evaluation includes results for the LLM DeepSeek-Coder-V2. Examining DeepSeek's GitHub repo suggests that there are two possible models this could refer to: a 16B and a 236B parameter model. Which one was used in the evaluation?

We have included both of them. You can refer to the leaderboard for details.