isl-org/PhotorealismEnhancement

Trouble in training with fake gbuffers

WHPiKA opened this issue · 1 comments

Hi, since my own dataset is still in process, recently I tried to train the model with pfd data using fake gbuffers extracted from vgg.
And I got the following results when achieved 4k and 8k iterations.
4000_02407
8000_02407
Is there any idea about the wrong results? Such a large difference between gbuffer and features from vgg?
By the way, I trained the model about 2 iterations per minute. I'm not sure the reason is the increase of number of channels or other things. The generator took too long to forward.
Thanks for any help

The fake G-buffer implementation is just a placeholder to outline how G-buffers need to be fed into the network. I would not expect any big effects from the fake G-buffers. That said, the results look very weird with the artifacts on top - it looks like the discriminator is way too strong. The adaptive backpropagation should largely prevent this. Did you change the learning rate? If you continue to see this, I'd recommend trying out the following (individually): increasing the regularization of the discriminator, increasing the weight of the VGG-loss, and reducing the capacity of the discriminator by reducing the number of channels or layers.