lucidrains/ema-pytorch

Question about EMA

slerman12 opened this issue · 1 comments

slerman12 commented

Is the EMA model always independent from the main model or does the main model periodically get updated too with the EMA model?

lucidrains commented

Yup, it is always independent

This EMA is good for generative models, and for other works where you use the output of the EMA as part of the loss (byol, https://arxiv.org/abs/2205.14083), but probably not for what you are thinking of

  • 🎉1