query about freeze BN from beginning

Question

query about freeze BN from beginning

Torment123 opened this issue 2 years ago · 3 comments

Hi,
I found that in the training all the batchnorm2d layer are frozen from the very beginning, indicating that these set of parameters are randomly initialized without any learning, I'm a bit confused of why this step is needed? Thanks.

Answer 1 · 2022-08-30T16:54:51.000Z

I believe batchnorm2d is initialized with mean 0 and variance 1 by default. We freeze the batchnorm2d parameters because empirically this worked well, although the difference was negligible.

Answer 2 · 2022-08-30T17:53:25.000Z

Thanks for your fast response.

Answer 3 · 2022-08-30T17:53:38.000Z

Thanks for your fast response.