modelscope/FunCodec

LauraTTS的GPT没有任何优化吗? 直接用的torch.matmul?

MinJ-lucky opened this issue · 5 comments

MinJ-lucky commented

如题... 这样batchsize能撑起来吗?

ZhihaoDU commented

Sorry, I can't understand your question, please provide more details to specify the question, thanks.

MinJ-lucky commented

My bad, let me explain my question.
As far as I know, big GPT-like models always use some optimization in attention like FlashAttention for less GPU mem cost and improve the forward speed. However, I didn't find any like this, only find a "torch.matmul" at here. I'd like to make sure that I'm correct, or there is optimization somewhere else~

ZhihaoDU commented

Yes, you are correct. In FunCodec and LauraTTS, we didn't include any optimization on attention operation, since the model is very small.

  • 👍1
MinJ-lucky commented

Thanks for your reply!
After reading the code, I still have two little questions:

  1. Is LauraTTS a totally independent model, which is trained from scratch without any pretrained parameters from LauraGPT?
    2.Since I see LauraGPT paper is also referenced in this repo, also taking into consideration that LauraGPT and LauraTTS has highly similar name, what's the relevance between both?
ZhihaoDU commented

Thanks for your reply! After reading the code, I still have two little questions:

  1. Is LauraTTS a totally independent model, which is trained from scratch without any pretrained parameters from LauraGPT?
    2.Since I see LauraGPT paper is also referenced in this repo, also taking into consideration that LauraGPT and LauraTTS has highly similar name, what's the relevance between both?
  1. Yes, LauraTTS is a totally independent model, you can train it from scratch on LibriTTS corpus with the released code.
  2. The reason of LauraGPT paper referencing this repo is because LauraGPT use the same codec model to tokenize speech signals. In fact, LauraTTS can be treated as a improved version of LauraGPT for TTS task only. There are two main differences between LauraTTS and LauraGPT in terms of TTS task: 1) LauraTTS use two codec groups in LM model rather than one. 2) In LauraTTS the LM and NAR model are jointly trained while they are trained separately in LauraGPT.