Results of MNIST and CIFAR10

[bdseal]

Could you please provide the targeted and untargeted results of MNIST and CIFAR-10？

[max-andr]

Hi bdseal,

We already present results on these datasets in our paper: on Madry and TRADES models in Table 4 (MNIST) and on CLP and LSQ models in Table 6 (MNIST and CIFAR-10). Although in these experiments we focus only on untargeted attacks.

If you would like to evaluate Square Attack on some other models or additionally benchmark query efficiency, please check our instructions on how to run our code: https://github.com/max-andr/square-attack#running-the-code. It's really simple:

python attack.py --attack=square_linf --model=madry_mnist_robust --n_ex=10000 --eps=0.3 --p=0.8 --n_iter=20000

And you can add the --targeted flag to make the attack targeted (against a randomly chosen target class).

I hope that helps!

Best,
Maksym

[bdseal]

thanks a lot.