├── data
└── clean_validation_data.h5 // this is clean data used to evaluate the BadNet and design the backdoor defense
├── models
└── bd_net.h5
└── bd_weights.h5
├── architecture.py
└── eval.py // this is the evaluation script
- Python 3.6.9
- Keras 2.3.1
- Numpy 1.16.3
- Matplotlib 2.2.2
- H5py 2.9.0
- TensorFlow-gpu 1.15.2
- Download the clean validation data from here and store the data in
data/clean_validation_data.h5
directory. - The dataset contains images from YouTube Aligned Face Dataset. We retrieve 883 individuals each containing 10 images in the validation dataset.
-
The DNN architecture used to train the face recognition model is the state-of-the-art DeepID network. This DNN is backdoored with multiple triggers. Each trigger is associated with its own target label. Essentially, the provided BadNet is a Multi-Trigger Multi-Target setting.
-
To evaluate the backdoored model, execute
eval.py
by running:
python3 eval.py <clean validation data directory> <model directory>
.E.g.,
python3 eval.py data/clean_validation_data.h5 models/bd_net.h5
. -
Clean data classification accuracy on the provided validation dataset is 96.07 %.
To aid teams in designing their defense, here are a few guidelines to keep in mind to get maximum points for the submission:
- Defense should generalize well to other backdoored networks. To verify the defense generalizability, the organizers will evaluate the submission on a specially curated BadNet with different trigger properties.
- Teams gain maximum points if the defense greatly reduces attack success rate on the trigger(s) while maintaining high clean classification accuracy.
- Points will also be given to teams that identify poisoned images in the online test stream of images.
- Fewer points will be allocated to teams that only detect the network as clean or backdoored.
- Report should contain a description of the defense performance on adaptive attackers.