A Training Bug
zjykzj opened this issue ยท 2 comments
zjykzj commented
There is a training bug in this project. That is I only set teacher model's require_grad_=False, but still put it's parameters to optimizer. So teacher model will update in training while it doesn't compute grad.
I don't have plan to fix it, because the training result shows it also works well.
zjykzj commented
May be that's fine if I had set requires_grad=False. See What is the behavior of passing model parameters with requires_grad == False
to an optimizer?
Optimizer has done the filter func implicitly, for example sgd. To be on the safe side, filter the parameters entered into the optimizer
thancaocuong commented
it's not a bug ๐