bytedance/SALMONN

The role of prompt_pattern parameter

Closed this issue · 2 comments

980202006 commented

I noticed that there is this parameter in your code: prompt_pattern. For music, do I need to modify it? Can you briefly talk about the process of training this model and the data set used?

TCL606 commented

You dont need to modify the prompt_pattern as this is the format used for all tasks during training.

For the training process, we first trained SALMONN using large amount of speech recognition and audio captioning data, then applied multi-task instruction finetuning on it. All the data for training is open source. We are planning to release the paper of SALMONN very soon, so stay tuned ~

980202006 commented

Thank you for your reply. This is a great work.

  • 👍1