query about freeze BN from beginning
Torment123 opened this issue · 3 comments
Torment123 commented
Hi,
I found that in the training all the batchnorm2d layer are frozen from the very beginning, indicating that these set of parameters are randomly initialized without any learning, I'm a bit confused of why this step is needed? Thanks.
lahavlipson commented
I believe batchnorm2d is initialized with mean 0 and variance 1 by default. We freeze the batchnorm2d parameters because empirically this worked well, although the difference was negligible.
Torment123 commented
Thanks for your fast response.
Torment123 commented
Thanks for your fast response.