Beckschen/TransUNet

PreActBottleneck

Opened this issue · 0 comments

why did the author use PreActBottleneck as the name , but in the code it used GroupNorm and Relu after Convolution?

def forward(self, x):
    # Residual branch
    residual = x
    if hasattr(self, 'downsample'):
        residual = self.downsample(x)
        residual = self.gn_proj(residual)

    # Unit's branch
    y = self.relu(self.gn1(self.conv1(x)))
    y = self.relu(self.gn2(self.conv2(y)))
    y = self.gn3(self.conv3(y))

    y = self.relu(residual + y)
    return y