ResNet First Conv Layer
aminullah6264 opened this issue · 1 comments
aminullah6264 commented
I noticed that the first conv layer of ResNet is followed by BatchNorm Layer, I think it should also be followed by EvoNorm as well.
self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False) self.bn1 = nn.BatchNorm2d(64) self.relu = nn.ReLU(inplace=True)
digantamisra98 commented
@aminullah6264 apologies, hadn't checked this issue. Yes you can absolutely use Evonorm for the input layer as well. I just decided to replace the batch normalisation layers in only the residual blocks.