Why L2 norm attack samples the same window over one-batch's images?
machanic opened this issue · 5 comments
In L-inf norm attack, square attack randomly samples different window positions for each image inside a mini-batch.
However, in L2 norm attack, square attack uses the same window position for different images of a mini-batch.
As shown in https://github.com/max-andr/square-attack/blob/master/attack.py#L156
Why?
It doesn't really matter since each image leads to a separate optimization problem. We can share randomness (i.e. in this case: window position) over multiple optimization problems that are solved in parallel, that's totally fine.
@max-andr Does it mean that L-inf norm attack can also sample the same window over different images inside a mini-batch, instead of sampling separately?
Yes, that would be totally fine for the Linf attack as well.
However, note that we decided to have a separate loop for Linf attack over each image:
https://github.com/max-andr/square-attack/blob/master/attack.py#L226
since it can happen (especially at later stages of the optimization process when the squares are small) that we sample an update which doesn't change the perturbation, and thus we would waste a query for some images. To prevent this, we iterate over each image separately and resample the color of the square-shaped update until we ensure that it leads to a new perturbation. Since we anyway had to implement a loop here, we also sample separately the window position for each image. But this is anyway not an important part.
@max-andr How to set epsilon in CIFAR-10 dataset (32x32x3) for L2 norm attack? Is the number of 5 too large?
For L2-norm, eps=0.5 is a reasonable choice, e.g., see RobustBench: a standardized adversarial robustness benchmark: a standardly trained model has 0% robust accuracy under eps=0.5 perturbations.
