Fail to reproduce the reconstruction result
kebijuelun opened this issue · 1 comments
kebijuelun commented
Thank you for the excellent open-source work.
- reproduce the reconstruction result （ImageNet dataset） （refer to Table 2 from https://arxiv.org/pdf/2312.03511.pdf ）
| FID ↓ | SSIM ↑ | PSNR ↑ | |
|---|---|---|---|
| from paper | 0.686 | 0.741 | 27.04 |
| reproduce | 1.19 | 0.76 | 24.8 |
Is this kind of accuracy difference as expected?
kebijuelun commented
-
I found that there are some difference between https://github.com/ai-forever/MoVQGAN/blob/main/movqgan/models/vqgan.py and this repo
- movq of this repo did not implement vector quantize in https://github.com/ai-forever/Kandinsky-3/blob/main/kandinsky3/movq.py#L402
- some norm layer in encoder have different weight and bias
-
so I inference the movq from https://github.com/ai-forever/MoVQGAN
| FID ↓ | SSIM ↑ | PSNR ↑ | |
|---|---|---|---|
| from paper | 0.686 | 0.741 | 27.04 |
| reproduce movqgan_270M (256x256) | 1.08 | 0.75 | 27.42 |
| reproduce movqgan_270M (512x512) | 0.695 | 0.792 | 26.32 |
when inference with 512x512, The result looks somewhat closer to the paper