This repository is used to record the defense result of our team, trained by TRADESv2 on ResNet-50 (without using ImageNet adversarial pretraining). The checkpoint does not represent the best performance limit of TRADESv2.
All percentages above correspond to the model's accuracy at 80% coverage.
| Defense | Submitted by | Clean data | Common corruptions | Spatial grid attack | SPSA attack | Boundary attack | Submission Date |
|---|---|---|---|---|---|---|---|
| Pytorch ResNet50 (trained on bird-or-bicycle extras) |
Hongyang Zhang (CMU) & Xin Li (Lehigh Univ.) | 100.0% | 100.0% | 99.5% | 100.0% | 95.0% | Jan 17th, 2019 (EST) |
| Keras ResNet (trained on ImageNet) |
Google Brain | 100.0% | 99.2% | 92.2% | 1.6% | 4.0% | Sept 29th, 2018 |
| Pytorch ResNet (trained on bird-or-bicycle extras) |
Google Brain | 98.8% | 74.6% | 49.5% | 2.5% | 8.0% | Oct 1st, 2018 |
-
Step 1: Clone and install the dependencies following the instructions: https://github.com/google/unrestricted-adversarial-examples/tree/master/bird-or-bicycle
-
Step 2: Download our evaluation code:
git clone https://github.com/xincoder/google_attack.git
-
Step 3: Download our pre-trained weight: https://drive.google.com/file/d/1l7uZW73gMzwvBDR5WWOXVPY1vWX3WEk4/view?usp=sharing and put it into the folder "google_attack"
-
Step 4: Run the code:
python eval_hongyangxin.py