The results on COCO seem weird
fortunechen opened this issue · 6 comments
As I mention in #4
I also get very weird results on COCO. And I test the results of the released model on COCO, which only gets FID score of 80! Any explanation?
@Corleone-Huang
I have a similar problem too. I have doubted it was because of my incorrect application of the previous code.... What about your other results, IS and R-precision.
Very sorry for the confusion and inconvenience. For model running in the first time, we need to choose the grammar rule(for COCO or CUB) in load_captions() function in dataset.py. Thanks for your valuable comments, we have uploaded the preprocessed captions.pickle file and added related instructions for convenience based on your suggestions.
For some generated images in COCO showing poor visual effects, we guess the reason is that in each step, an aspect is added for enhancement, while they don't have accurate supervision signals. Therefore, some aspects might be added to the inappropriate positions in different steps.
Meanwhile, as both text and image have been pretrained jointly for image and text encoders, if we add aspect text information in each step, the relevant aspect visual information will also be added correspondingly. Since the corresponding key local aspect information has been contained, when calculating the relevant indicators, such as the matching degree of image and text, the difference between the real image and the generated image, we guess the quantitative results will also be not bad as analyzed.
Thx for your reply. I can reproduce the result now.
Thx for your reply. I also reproduce the results.
@Corleone-Huang I have a similar problem too. I have doubted it was because of my incorrect application of the previous code.... What about your other results, IS and R-precision.
The confusion lies in code/datasets.py line 173 - line 176.