Generative Adversarial Perturbations (GAP)

Code for the paper "Generative Adversarial Perturbations", CVPR 2018.

Usage

Separate files are used for generating perturbations for classification and segmentation models.

Classification

First specify the paths to both training and validation folders for ImageNet --imagenetTrain and --imagenetVal. We use the class IDs listed here.

Now we can run the following to start training a generative network for universal perturbations with target 805 (soccer ball):

CUDA_VISIBLE_DEVICES=0,1 python GAP_clf.py \
--expname test_incv3_universal_targeted_linf10_twogpu \
--batchSize 30 --testBatchSize 16 --mag_in 10 --foolmodel incv3 --mode train \
--perturbation_type universal --target 805 --gpu_ids 0,1 --nEpochs 10

Note the CUDA_VISIBLE_DEVICES flag is set to make both GPUs 0 and 1 visible. After the above model is done running, you can test a checkpoint like so (set --checkpoint to your model's checkpoint, change --mode train to --mode test, and set --MaxIterTest <number of test iters> sufficiently high):

CUDA_VISIBLE_DEVICES=0,1 python GAP_clf.py \
--expname test_incv3_universal_targeted_linf10_twogpu \
--batchSize 30 --testBatchSize 16 --mag_in 10 --foolmodel incv3 --mode test \
--perturbation_type universal --target 805 --gpu_ids 0,1 --nEpochs 10 --MaxIterTest 1700

Testing Universal Perturbation Tensor

If you have trained a universal perturbation model and want to just test one of the output perturbation tensors in the U_out folder of your experiment folder, you can specify it explicitly with explicit_U and just test without U noise or a checkpiont:

CUDA_VISIBLE_DEVICES=0,1 python GAP_clf.py \
--expname test_incv3_universal_targeted_linf10_twogpu_saving_U \
--testBatchSize 16 --mag_in 10 --foolmodel incv3 --mode test \
--perturbation_type universal --target 805 --gpu_ids 0,1 \
--explicit_U images_test_incv3_universal_targeted_linf10_twogpu_saving_U/U_out/U_epoch_10_top1target_54.54192546583851.pth

Full usage details are given below. You can similarly generate image-dependent perturbations, and make the perturbation non-targeted by specifying --target to be -1.

usage: GAP_clf.py [-h] [--imagenetTrain IMAGENETTRAIN]
                  [--imagenetVal IMAGENETVAL] [--batchSize BATCHSIZE]
                  [--testBatchSize TESTBATCHSIZE] [--nEpochs NEPOCHS]
                  [--ngf NGF] [--optimizer OPTIMIZER] [--lr LR]
                  [--beta1 BETA1] [--threads THREADS] [--seed SEED]
                  [--MaxIter MAXITER] [--MaxIterTest MAXITERTEST]
                  [--mag_in MAG_IN] [--expname EXPNAME]
                  [--checkpoint CHECKPOINT] [--foolmodel FOOLMODEL]
                  [--mode MODE] [--perturbation_type PERTURBATION_TYPE]
                  [--target TARGET] [--gpu_ids GPU_IDS]
                  [--path_to_U_noise PATH_TO_U_NOISE]
                  [--explicit_U EXPLICIT_U]

generative adversarial perturbations

optional arguments:
  -h, --help            show this help message and exit
  --imagenetTrain IMAGENETTRAIN
                        ImageNet train root
  --imagenetVal IMAGENETVAL
                        ImageNet val root
  --batchSize BATCHSIZE
                        training batch size
  --testBatchSize TESTBATCHSIZE
                        testing batch size
  --nEpochs NEPOCHS     number of epochs to train for
  --ngf NGF             generator filters in first conv layer
  --optimizer OPTIMIZER
                        optimizer: "adam" or "sgd"
  --lr LR               Learning Rate. Default=0.002
  --beta1 BETA1         beta1 for adam. default=0.5
  --threads THREADS     number of threads for data loader to use
  --seed SEED           random seed to use. Default=123
  --MaxIter MAXITER     Iterations in each Epoch
  --MaxIterTest MAXITERTEST
                        Iterations in each Epoch
  --mag_in MAG_IN       l_inf magnitude of perturbation
  --expname EXPNAME     experiment name, output folder
  --checkpoint CHECKPOINT
                        path to starting checkpoint
  --foolmodel FOOLMODEL
                        model to fool: "incv3", "vgg16", or "vgg19"
  --mode MODE           mode: "train" or "test"
  --perturbation_type PERTURBATION_TYPE
                        "universal" or "imdep" (image dependent)
  --target TARGET       target class: -1 if untargeted, 0..999 if targeted
  --gpu_ids GPU_IDS     gpu ids: e.g. 0 or 0,1 or 1,2.
  --path_to_U_noise PATH_TO_U_NOISE
                        path to U_input_noise.txt (only needed for universal)
  --explicit_U EXPLICIT_U
                        Path to a universal perturbation to use

Segmentation

Training

First download the Cityscapes dataset and then downsample the images and label maps to 1024x512 using bilinear and nearest-neighbor interpolation, respectively (augmentation). Next either download the pretrained model (FCN-8s trained to segment Cityscapes) or provide your own pretrained model on Cityscapes. Then run GAP_seg.py with suitable arguments. Specify the root folder in which the cityscapes dataset is stored using --dataroot and the pretrained model's path with --pretrained_cityscapes. If you are running a targeted experiment, you need to specify a target label map using --target_path.

You can check python GAP_seg.py -h for descriptions of all arguments.

Examples

Training a model with resnet generator:
- CUDA_VISIBLE_DEVICES=0,1 python GAP_seg.py -g 'resnet' --eps 10 --name 'test' --gpu_ids '0,1' --ngf 200 --lr 0.0001 --metric 0 --task 3 --display_freq 10 --block 5 --dataset cityscapes
Training a model with unet generator:
- CUDA_VISIBLE_DEVICES=0,1 python GAP_seg.py -g 'unet' --eps 10 --name 'test' --gpu_ids '0,1' --ngf 200 --lr 0.0001 --metric 2 --task 2 --dataset cityscapes

Tasks:

We specify tasks by different numbers:

Task Mapping
- 0 --- Non-targeted Image Dependent Task
- 1 --- Targeted Image Dependent Task
- 2 --- Non-targeted Universal Task
- 3 --- Targeted Universal Task

We use 'Mean IoU' for non-targeted tasks and 'Success Rate' for targeted tasks:

Metric Mapping
- 0 --- Success Rate (for task 1 and 3)
- 2 --- Mean IoU (for task 0 and 2)

Architecture

Universal Perturbations:

Image-dependent Perturbations:

Abstract

In this paper, we propose novel generative models for creating adversarial examples, slightly perturbed images resembling natural images but maliciously crafted to fool pre-trained models. We present trainable deep neural networks for transforming images to adversarial perturbations. Our proposed models can produce image-agnostic and image-dependent perturbations for targeted and nontargeted attacks. We also demonstrate that similar architectures can achieve impressive results in fooling both classification and semantic segmentation models, obviating the need for hand-crafting attack methods for each task. Using extensive experiments on challenging high-resolution datasets such as ImageNet and Cityscapes, we show that our perturbations achieve high fooling rates with small perturbation norms. Moreover, our attacks are considerably faster than current iterative methods at inference time.

Citation

If you use the code in this repository in your paper, please consider citing:

@InProceedings{Poursaeed_2018_CVPR,
  author = {Poursaeed, Omid and Katsman, Isay and Gao, Bicheng and Belongie, Serge},
  title = {Generative Adversarial Perturbations},
  booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  month = {June},
  year = {2018}
}

YanghaoZYH/Generative_Adversarial_Perturbations