TIGER-AI-Lab/MMLU-Pro

Why dont use chat template for chat model?

Closed this issue · 1 comments

It is an uncommon approach to directly input a prompt string into a chat model without using a chat template, which may result in unexpected behavior.

We tried with chat template, the performance is lower for few-shot learning. The chat template are mostly used in zero-shot for some models.