Fine-Tuning llama2-7b-chat model

[Beetavirusss]

I want to train my model on custom dataset to make a chatbot, how am I suppose to structure my dataset for this llama2-7B-chat model

[research4pan]

Thanks for your interest in LMFlow! As llama2-7B-chat is a decoder-only model, you may use "text-only" typed dataset to construct your own dataset. The "text-only" samples can contain certain structures such as "#Question: ... #Answer: ...#" so that the fine-tuned model can recognize the input, output and end mark from the structure. Hope that helps 😄