Adversarial Attack by Shadows

This repository contains the code of our CVPR 2022 paper "Shadows can be Dangerous: Stealthy and Effective Physical-world Adversarial Attack by Natural Phenomenon", which study a new type of optical adversarial examples, in which the perturbations are generated by a very common natural phenomenon, shadow, to achieve naturalistic and stealthy physical-world adversarial attack under the black-box setting.

(Our presentation video)

Structure

├── adv_img            // The directory for storing adversarial examples generated by shadow_attack.py
│   ├── GTSRB          // Adversarial examples from GTSRB dataset
│   └── LISA           // Adversarial examples from LISA dataset
│       └── 43         // Adversarial examples when k = 0.43
│           └── 81_10_43_True.bmp
│                      // 81: image index in the testing set; 10: ground truth class; 43: number of queries; True: attack succeed
├── dataset            
│   ├── GTSRB  
│   │   ├── train.pkl  // GTSRB training set  
│   │   └── test.pkl   // GTSRB testing set
│   └── LISA
|       ├── train.pkl  // LISA training set
│       └── test.pkl   // LISA testing set
├── mask               // Mask files for different types of traffic signs
│   ├── circle.pkl
│   ├── inverse_triangle.pkl
│   ├── octagon.pkl
│   ├── pentagon.pkl
│   ├── rectangle.pkl
│   ├── rhombus.pkl
│   └── triangle.pkl
├── model        
│   ├── adv_model_gtsrb.pth    // Adversarially trained model for GTSRB dataset
│   ├── adv_model_lisa.pth     // Adversarially trained model for LISA dataset
│   ├── model_gtsrb.pth        // Common model for GTSRB dataset
│   └── model_lisa.pth         // Common model for LISA dataset
├── tmp
├── gtsrb.py           // Code to train and test a model on the GTSRB dataset
├── lisa.py            // Code to train and test a model on the LISA dataset
├── params.json        // Some parameters
├── pso.py             // Code to implement particle swarm optimization
├── README.md
├── schedule_attack.py // The simulation of scheduled attack
├── shadow_attack.py   // Digital & physical attack by shadows
└── utils.py           // Some utility functions

Usage of shadow_attack.py

Datasets and trained models
You should first download the LISA and GTSRB datasets and our trained models and place them in dataset/ and model/, respectively.
Requirements:
```
python = 3.8.11
pytorch = 1.9.0
torchvision = 0.10.0
```
To run in an environment without cuda enabled, change "device": "cuda:0" to "device": "cpu" in params.json.

Example 1: show help message.

$ python3 shadow_attack.py --help

usage: shadow_attack.py [-h] [--shadow_level SHADOW_LEVEL] [--attack_db ATTACK_DB] [--attack_type ATTACK_TYPE] [--image_path IMAGE_PATH] [--mask_path MASK_PATH] [--image_label IMAGE_LABEL] [--polygon POLYGON] [--n_try N_TRY] [--target_model TARGET_MODEL]

optional arguments:
-h, --help           show this help message and exit
--shadow_level       shadow coefficient k
--attack_db          the target dataset should be specified for a digital attack
--attack_type        digital attack or physical attack
--image_path         the file path to the target image should be specified for a physical attack
--mask_path          the file path to the mask should be specified for a physical attack
--image_label        the ground truth should be specified for a physical attack
--polygon            The number of sides of polygon P.
--n_try              n-random-start strategy: retry n times
--target_model       attack normal model or robust model

Example 2: our digital attack:
The following shell will launch our digital attack on LISA while setting the shadow level $k$ as 0.43. The generated digital adversarial examples will be saved to adv_img/LISA/43/.

$ python3 shadow_attack.py --shadow_level 0.43 --attack_db LISA

try 1: Best solution: 0.09873462468385696 succeed
try 1: Best solution: 0.08393772691488266 succeed
try 1: Best solution: 0.053570982068777084 succeed
try 1: Best solution: 0.0678005963563919 succeed
...
try 1: Best solution: 0.09404987096786499 succeed
try 1: Best solution: 0.06214200705289841 succeed
try 1: Best solution: 0.11059863865375519 succeed
Attack finished! Success rate: 0.9822888283378747

Example 3: our physical attack:
The following shell will launch our physical attack. The generated images will be saved as ./tmp/adv_img.png.

$ python3 shadow_attack.py --shadow_level 0.43 --attack_db GTSRB --attack_type physical --image_path ./tmp/gtsrb_30.png --mask_path ./tmp/gtsrb_30_mask.png --image_label 1

iteration: 1 0.7416303753852844
iteration: 2 0.7226707339286804
iteration: 3 0.7226707339286804
iteration: 4 0.7226707339286804
iteration: 5 0.6017401814460754
...
iteration: 198 0.07176369428634644
iteration: 199 0.07176369428634644
iteration: 200 0.07176369428634644
Best solution: 0.9282363057136536 succeed
Correct: False Predict: 40 Confidence: 90.27248024940491%
Attack succeed! Try to implement it in the real world.

Example 4: Change the number of sides of polygon P, e.g., 4:
```
$ python3 shadow_attack.py --attack_db LISA --polygon 4
```

hncszyq/ShadowAttack

Adversarial Attack by Shadows

Structure

Usage of shadow_attack.py