Mixup Inference

This repository contains the codes for reproducing most of the results of our paper.

Paper Tittle:

Mixup Inference: Better Exploiting Mixup to Defend Adversarial Attacks (ICLR 2020)

Tianyu Pang*, Kun Xu* and Jun Zhu

Environment settings and libraries we used in our experiments

This project is tested under the following environment settings:

OS: Ubuntu 16.04.3
GPU: Geforce 1080 Ti or Tesla P100
Cuda: 10.0, Cudnn: v7.4
Python: 3.5.2
PyTorch: 1.2.0
Torchvision: 0.4.0

For convenience, we provide the requ.txt file to install the virtualenv that is sufficient run the codes.

In the following, we first provide the codes for training with mixup and interpolated AT. After that, the evaluation codes, such as attacking our mixup inference (MI) method and other baselines, are provided.

Training codes

Training with the mixup mechinism

Let dataset be cifar10 or cifar100, the command for training models with the mixup mechanism is

python train_resnet_mixup.py -model resnet50 -lr 0.01 -adv_ratio 0. -mixup_alpha 1. -data [dataset] -bs 64

When applying mixup, the initial learning rate is 0.01, the alpha is 1.0, the optimizer is mom and we use the ResNet-50 architecture proposed by He et al. (2016). The training epoch on both CIFAR-10 and CIFAR-100 is set as 200. Pretrained models are avaiable: mixup checkpoint (CIFAR-10) and mixup checkpoint (CIFAR-100).

Training with the interpolated AT mechinism

The command for training models with the interpolated AT mechanism is

python train_resnet_mixup.py -model resnet50 -lr 0.1 -adv_ratio 0.5 -mixup_alpha 1. -data [dataset] -bs 64

When applying interpolated AT, the initial learning rate is 0.1, the alpha is 1.0, the optimizer is mom. The ratio between the clean samples and the adversarial ones is 1:1. Pretrained models are avaiable: IAT checkpoint (CIFAR-10) and IAT checkpoint (CIFAR-100).

Evaluation codes

We mainly evluate the PGD attacks for different inference-phase defenses. Here the eps is by default 8/255, the pixels are normalized to the interval [-1,1]. About the detailed parameter settings to re-implement the results in our paper, please refer to Table 4 and Table 5 in our appendix.

Evaluating MI-PL

Let dataset be cifar10 or cifar100. The model_checkpoint be the file of trained model checkpoint, which could be trained by mixup or interpolated AT or other training methods. The evaluation command is

python attack_resnet_mixuptest_PL.py -targeted=False -nbiter=10 -data=[dataset] -model=resnet50 -oldmodel=model_checkpoint/model.ckpt -lamda=0.5

The FLAG targeted indicates whether use targeted attacks or untargeted attacks. nbiter is the iterations steps of the PGD attacks. For exmaple, the attack here is untargeted PGD-10. Here lamda is the mixup ratio for MI-PL.

Evaluating MI-OL

Similar to the command for MI-PL, the one for evaluating MI-OL is

python attack_resnet_mixuptest_OL.py -targeted=False -nbiter=10 -data=[dataset] -model=resnet50 -oldmodel=model_checkpoint/model.ckpt -lamda=0.5

Here lamda is the mixup ratio for MI-OL, where we use lamda=0.5 for mixup+MI-OL and lamda=0.6 for IAT+MI-OL.

Evaluating MI-Combined

The command is

python attack_resnet_mixuptest_Combined.py -targeted=False -data=[dataset] -model=resnet50 -oldmodel=model_checkpoint/model.ckpt -lamdaPL=0.5 -lamdaOL=0.4 -threshold=0.2

Here lamdaPL is the lambda for MI-PL, lamdaOL is the lambda for MI-OL, threshold is used to decide whether the input is adversarial according to the detection result returned by MI-PL.

Evaluating Baselines

The command for evaluating Gaussian noise baseline is

python -u attack_resnet_baselines.py -targeted=False -nbiter=10 -data=[dataset] -eps=8 -model=resnet50 -oldmodel=model_checkpoint/model.ckpt -baseline='gaussian' -num_sample=50 -sigma=0.05 -numtest=1000

The command for evaluating Rotation baseline is

python -u attack_resnet_baselines.py -targeted=False -nbiter=10 -data=[dataset] -eps=8 -model=resnet50 -oldmodel=model_checkpoint/model.ckpt -baseline='Rotation' -num_sample=50 -Rotation=20 -numtest=1000

The command for evaluating Xie et al. (2018) baseline is

python -u attack_resnet_baselines.py -targeted=False -nbiter=10 -data=[dataset] -eps=8 -model=resnet50 -oldmodel=model_checkpoint/model.ckpt -baseline='Xie' -num_sample=50 -Xielower=24 -Xieupper=32 -numtest=1000

The command for evaluating Guo et al. (2018) baseline is

python -u attack_resnet_baselines.py -targeted=False -nbiter=10 -data=[dataset] -eps=8 -model=resnet50 -oldmodel=model_checkpoint/model.ckpt -baseline='Guo' -num_sample=50 -Guolower=24 -Guoupper=32 -numtest=1000

Evaluating Adaptive Attacks for MI-OL

The adaptive attacks are perform under different number of adaptive samples, as shown in the command below

x=( 1 2 3 4 5 10 15 20 25 30 )
for num_sample in ${x[@]}
do
    python attack_resnet_mixuptest_OL_adaptive.py -targeted=False -nbiter=10 -data=[dataset] -model=resnet50 \
    -oldmodel=model_checkpoint/model.ckpt \
    -lamda=0.5 -adaptive=True -adaptive_num=$num_sample;
done

This is an example of adaptive PGD-10 attack.

P2333/Mixup-Inference