[BUG] ActNorm returns NaN

[mjack3]

Hello.

I am getting NaN using this arquitecture

ActNorm + PermuteRandom + GLOWCouplingBlock

The reason is that the mean and standard deviation of ActNorm are initialized from the first batch of data passed to them. As I have at least one channel which always has the same value, ActNorm leads to division by 0 (std = 0).

Then, I added Gaussian Noise but I am still getting NaN values.

from albumentation.pytorch import TensorV2
import Albumentations as A

training_transforms = A.Compose([
            A.Resize(resize, resize),
            A.CenterCrop(crop, crop),
            A.GaussNoise(p=1.),
            A.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)),
            ToTensorV2(),
        ])

Is this behaviour expected?
#113

[robertfirmo]

Hi, this behaviour is not expected - in the future, initializing the actNorm layer with zero std will raise an error, but adding noise should avoid this problem. could you verify that the noise is added (did you maybe add the same noise for the whole batch?) maybe try adding noise manually to see if the problem still occurs.

[robertfirmo]

we added a line of code to test for nans while initializing, do you get an error while running the updated code?

[psorrenson]

we added a line of code to test for nans while initializing, do you get an error while running the updated code?

See #120

[mjack3]

I will test it later
Thanks