Reverse attack on classic metrics
ZhengyuZhao opened this issue · 2 comments
Nice work, especially the discussion of reverse attack in Appendix F! As I understand, the reverse attack makes more sense because it causes the conflict between the model and human decisions: distorting the image to make it cleaner (i.e., more similar to the original image).
So I am very curious whether the reverse attack also works on classic metrics, e.g., SSIM.
Thank you. Yes, the reverse attack, i.e., the attack that makes images more similar, does make more sense. Traditional metrics such as SSIM are susceptible to it in the same way they are susceptible to the attack that makes images less similar.
When I perform the whitebox PGD attack that makes images more similar, I get the following results:
Samples where rank by human and metric was the same (or accurate samples): 9883
Samples where rank by human and metric was not the same (or not accurate samples): 2344
Accurate samples that flipped after attack: 5032
Not accurate samples that flipped after attack: 1855
Thank you. Yes, the reverse attack, i.e., the attack that makes images more similar, does make more sense. Traditional metrics such as SSIM are susceptible to it in the same way they are susceptible to the attack that makes images less similar.
When I perform the whitebox PGD attack that makes images more similar, I get the following results:
Samples where rank by human and metric was the same (or accurate samples): 9883 Samples where rank by human and metric was not the same (or not accurate samples): 2344 Accurate samples that flipped after attack: 5032 Not accurate samples that flipped after attack: 1855
Thanks for your quick reply! That's indeed an impressive result.