Concerns about evaluation methods in the paper

[kplum]

Thanks for your contribution, @ZPdesu, but I have some questions about your paper.

1. In this comment (#7 (comment)) you say that you compute the FID between generated images from the test set and real images from the training set. Could you explain why you chose this approach? In my opinion this is not what we really want to measure, the networks might overfit to the training set and performance should always be measured using only unseen images.
2. Why are the pix2pixHD images for Cityscapes that are shown in the paper and with which you supposedly compare the SEAN method so bad? They look nothing like the pix2pixHD images from the original paper or what I have reproduced with pix2pixHD in smaller resolutions. Did you train pix2pixHD yourself, if so could you specify with what parameters?