Batch Norm doubts

[EmanueleGhelfi]

Hi,
I have some doubts regarding pix2pix.
In section 3.3:

"At inference time, we run the generator net in exactly the same manner as during the training phase. This differs from the usual protocol in that we apply dropout at test time,and we apply batch normalization [29] using the statistics of the test batch, rather than aggregated statistics of the training batch. This approach to batch normalization, when the batch size is set to 1, has been termed “instance normalization” and has been demonstrated to be effective at image generation tasks [54]. In our experiments, we use batch sizes between 1 and 10 depending on the experiment."

With batch size = 1 you can run your model in training mode even when testing since it uses only the current batch statistics. The BatchNormalization layer has only learned how to scale the activation of the previous layer.

With batch size = 10 during training how do you manage the testing? Do you run tests with batch size = 10 and training = True? I think this is not correct since the prediction for an example highly depends on the other examples in the batch. Do you run tests with batch size = 1? In this case is the output of your model correct?

I'm asking this since I'm having some issues with my implementation in tensorflow:

If I visualize the output of my model with the flag training=False the images are very bad. There are many artifacts. Visualizing the output with tranining=True is very different and images look good. Currently I'm training with batch_size=1 and training with batch size > 1 breaks all.