AttributeError: 'BatchNormFlow' object has no attribute 'batch_mean' in inverse mode

[zhouhanc]

I'm running into a problem when I call model.forward(..., mode='inverse') before calling model.forward(..., mode='direct'), in which case batch_mean and batch_var are not defined. What is a proper why to fix this problem? I need to use inverse mode first during the training stage in my application.

The code snippet looks like this. I've omitted some application specific codes.

block = [
fnn.MADE(num_inputs, num_hidden),
fnn.BatchNormFlow(num_inputs),
fnn.Reverse(num_inputs)
]
model = fnn.FlowSequential(*blocks)
model.train()
model.forward(..., mode='inverse')

AttributeError: 'BatchNormFlow' object has no attribute 'batch_mean'

[ikostrikov]

The easiest way to fix the problem would be to actually switch modes in BN.

Just replace 'direct' with 'inverse' here:
https://github.com/ikostrikov/pytorch-flows/blob/master/flows.py#L148