kyutai-labs/moshi

About llm

Closed this issue · 2 comments

wntg commented

Due diligence

  • I have done my due diligence in trying to find the answer myself.

Topic

The paper

Question

Why not use the existing llm and train it yourself from scratch?

Strive-for-excellence commented

I have the same problem,why not use the llama 7B.

adefossez commented

Llama has a restrictive license, similarly to a number of models released by GAFAM (e.g. Moshi would have to be called LlamaMoshi). Now there are some more alternatives, but they didn't exist when we started the project. Finally, we also have a co-training phase where we keep feeding text batches to the model, and for that we need a text training pipeline.

  • 👍1