Error in SoftGate code？

[isolution911]

the code at this line https://github.com/sebgao/LIP/blob/5e85b9e55b9212fdf6abccb1fa8783d23620f53a/imagenet/lip_resnet.py#L28 in SoftGate:
return torch.sigmoid(x).mul(COEFF)
maybe wrong, the correct code should be like this:
return x.mul(COEFF).sigmoid().

[sebgao]

It's return torch.sigmoid(x).mul(COEFF) although it looks weird. Multiplication after sigmoid is needed to provide the large value range of logits and hence more diverse pooling weights. But actually, it's OK to remove SoftGate when you do not meet NaN errors.

[isolution911]

It's return torch.sigmoid(x).mul(COEFF) although it looks weird. Multiplication after sigmoid is needed to provide the large value range of logits and hence more diverse pooling weights. But actually, it's OK to remove SoftGate when you do not meet NaN errors.

Now I am confused about this。If multiplying after sigmoid, the COEFF will be reduced during pooling procedure, which makes no sense。And the beta(COEFF) is in the exponent of weight，which is different from your code here。I am looking forward to getting your reply，thanks。

[sebgao]

It's return torch.sigmoid(x).mul(COEFF) although it looks weird. Multiplication after sigmoid is needed to provide the large value range of logits and hence more diverse pooling weights. But actually, it's OK to remove SoftGate when you do not meet NaN errors.

Now I am confused about this。If multiplying after sigmoid, the COEFF will be reduced during pooling procedure, which makes no sense。And the beta(COEFF) is in the exponent of weight，which is different from your code here。I am looking forward to getting your reply，thanks。

I'm sorry but there is no reduction about COEFF here because multiplication in the exponential operator appearing both in the numerator and denominator cannot cancel, e.g., exp(c*x)/exp(c*y)!=exp(x)/exp(y).

And for you another question, what does beta(*) stand for? The weight is the exponent of the logit. And there is no exponents of weight in either the paper or code.

[isolution911]

It's return torch.sigmoid(x).mul(COEFF) although it looks weird. Multiplication after sigmoid is needed to provide the large value range of logits and hence more diverse pooling weights. But actually, it's OK to remove SoftGate when you do not meet NaN errors.

Now I am confused about this。If multiplying after sigmoid, the COEFF will be reduced during pooling procedure, which makes no sense。And the beta(COEFF) is in the exponent of weight，which is different from your code here。I am looking forward to getting your reply，thanks。

I'm sorry but there is no reduction about COEFF here because multiplication in the exponential operator appearing both in the numerator and denominator cannot cancel, e.g., exp(c*x)/exp(c*y)!=exp(x)/exp(y).

And for you another question, what does beta(*) stand for? The weight is the exponent of the logit. And there is no exponents of weight in either the paper or code.

I'm so sorry for that I missed this line weight = logit.exp() when I read the code before, and I realized this just now after your reminding. I’m sorry again for bothering you beause of my carelessness，and appreciate your help greatly.