Issues reproducing results
NOlivier-H opened this issue · 2 comments
Hello, thanks for your work, and for making your code available.
I am having issues reproducing your results on CelebA, the network seems to converge up to around iteration 15k, but then the image quality degrades until becoming nothing recognizable.
I used the command python train_surf.py --output_dir exp_name --curriculum CelebA_single, with nothing changed from your code.
The python env was created using your instructions in "Create virtual environment".
I ran the training process 3 times, and consistently obtained this result.
Here are random samples at 15k, and 40k:
Hi @NOlivier-H , Thanks for you attention.
I just checked the train and curriculum files, I found that the batch size (now 14) is wrong.
For the first stage, I used the batch_size: 56 (not 14) At each stage, the batch size is multiplied by 1/4.
I forgot to change the original value after experimenting with my code under different conditions.
Therefore, please change the value of batch_size of the first stage in curriculums.py-CelebA_single.
Generally, with higher batch_size, you can expect higher quality. so set the maximum batch size for your GPU.
I changed batch_size of the first stage 14-->56