ViswanathaReddyGajjala/EfficientNet-RetinaNet

Question regarding Batch Norm

pasandrei opened this issue · 1 comments

pasandrei commented

Hello! I was wondering why are you setting the BatchNorm layers to eval()? I usually only set the weight decay to 0 for BN layers and biases so that those layers still learn.

ViswanathaReddyGajjala commented
  • I have used a pre-trained backbone. So, the network's batch norm layers have been trained.
  • In general, object detection's batch size is very small compared to classification model batch sizes. So, it is hard to stabilize the batch norm params.

Because of the above reasons, I have used the eval() mode for BatchNorm layers. I hope this answers your question.

  • 👍1