Quick question about input format of base and tuned models

[chujiezheng]

Thanks for your great work! I have a quick question about the input format of the base model and the tuned model, such as llama-2 vs. llama-2-chat. Concretely, llama-2-chat uses sequences like [INST], [/INST], <<<SYS>>> and <<</SYS>>> to distinguish the system prompt and user/assistant utterances. See [1], [2], while I think llama-2 may not understand these texts if directly prompted with the same format. So I wonder about the specific input formats of llama-2 and llama-2-chat (and if possible, other base/tuned models).

[yuchenlin]

Hi Chujie,

Thanks for the great question. We use the zero-shot prompting template for untuned llm and use the official template (as you mentioned) for the aligned LLMs. These details are shown in Appendix C on Page 23. And you can also use this web demos to see the concrete prefixes used for base and aligned LLMs: https://allenai.github.io/re-align/tds/llama2/

For example, if you click the token artists, then you can see the two prefixes which are used for inputing base and aligned LLMs respectively.

I hope this answers your question! Thanks!

Best,
Yuchen

[chujiezheng]

Thanks for your reply. It resolves my question!