Backdoor attacks are emerging yet critical threats in the training process of deep neural networks (DNNs), where the adversary intends to embed specific hidden backdoor into the models. The attacked DNNs will behave normally in predicting benign samples, whereas the predictions will be maliciously changed whenever the adversary-specified trigger patterns appear. Currently, there were many existing backdoor attacks and defenses. Although most of them were open-sourced, there is still no toolbox that can easily and flexibly implement and compare them simultaneously.
BackdoorBox is an open-sourced Python toolbox, aiming to implement representative and advanced backdoor attacks and defenses under a unified framework that can be used in a flexible manner. We will keep updating this toolbox to track the latest backdoor attacks and defenses.
Currently, this toolbox is still under development (but the attack parts are almost done) and there is no user manual yet. However, users can easily implement our provided methods by referring to the tests sub-folder to see the example codes of each implemented method. Please refer to our paper for more details! In particular, you are always welcome to contribute your backdoor attacks or defenses by pull requests!
- Consistency: Instead of directly collecting and combining the original codes from each method, we re-implement all methods in a unified manner. Specifically, variables having the same function have a consistent name. Similar methods inherit the same base class for further development, have a unified workflow, and have the same core sub-functions (e.g.,
get_model()). - Simplicity: We provide code examples for each implemented backdoor attack and defense to explain how to use them, the definitions and default settings of all required attributes, and the necessary code comments. Users can easily implement and develop our toolbox.
- Flexibility: We allow users to easily obtain important intermediate outputs and components of each method (e.g., poisoned dataset and attacked/repaired model), use their local samples and model structure for attacks and defenses, and interact with their local codes. The attack and defense modules can be used jointly or separately.
- Co-development: All codes and developments are hosted on Github to facilitate collaboration. Currently, there are more than seven contributors have helped develop the code base and others have contributed to the code test. This developing paradigm facilitates rapid and comprehensive development and bug finding.
| Method | Source | Key Properties | Additional Notes |
|---|---|---|---|
| BadNets | Badnets: Evaluating Backdooring Attacks on Deep Neural Networks. IEEE Access, 2019. | poison-only | first backdoor attack |
| Blended | Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning. arXiv, 2017. | poison-only, invisible | first invisible attack |
| Refool (simplified version) | Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks. ECCV, 2020. | poison-only, sample-specific | first stealthy attack with visible yet natural trigger |
| LabelConsistent | Label-Consistent Backdoor Attacks. arXiv, 2019. | poison-only, invisible, clean-label | first clean-label backdoor attack |
| TUAP | Clean-Label Backdoor Attacks on Video Recognition Models. CVPR, 2020. | poison-only, invisible, clean-label | first clean-label backdoor attack with optimized trigger pattern |
| SleeperAgent | Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch. NeurIPS, 2022. | poison-only, invisible, clean-label | effective clean-label backdoor attack |
| ISSBA | Invisible Backdoor Attack with Sample-Specific Triggers. ICCV, 2021. | poison-only, sample-specific, physical | first poison-only sample-specific attack |
| WaNet | WaNet - Imperceptible Warping-based Backdoor Attack. ICLR, 2021. | poison-only, invisible, sample-specific | |
| Blind (blended-based) | Blind Backdoors in Deep Learning Models. USENIX Security, 2021. | training-controlled | first training-controlled attack targeting loss computation |
| IAD | Input-Aware Dynamic Backdoor Attack. NeurIPS, 2020. | training-controlled, optimized, sample-specific | first training-controlled sample-specific attack |
| PhysicalBA | Backdoor Attack in the Physical World. ICLR Workshop, 2021. | training-controlled, physical | first physical backdoor attack |
| LIRA | LIRA: Learnable, Imperceptible and Robust Backdoor Attacks. ICCV, 2021. | training-controlled, invisible, optimized, sample-specific | |
| BATT | BATT: Backdoor Attack with Transformation-based Triggers. ICASSP, 2023. | poison-only, invisible, physical |
Note: For the convenience of users, all our implemented attacks support obtaining poisoned dataset (via .get_poisoned_dataset()), obtaining infected model (via .get_model()), and training with your own local samples (loaded via torchvision.datasets.DatasetFolder). Please refer to base.py and the attack's codes for more details.
| Method | Source | Defense Type | Additional Notes |
|---|---|---|---|
| AutoEncoderDefense | Neural Trojans. ICCD, 2017. | Sample Pre-processing | first pre-processing-based defense |
| ShrinkPad | Backdoor Attack in the Physical World. ICLR Workshop, 2021. | Sample Pre-processing | efficient defense |
| FineTuning | Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks. RAID, 2018. | Model Repairing | first defense based on model repairing |
| Pruning | Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks. RAID, 2018. | Model Repairing | |
| MCR | Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness. ICLR, 2020. | Model Repairing | |
| NAD | Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks. ICLR, 2021. | Model Repairing | first distillation-based defense |
| ABL | Anti-Backdoor Learning: Training Clean Models on Poisoned Data. NeurIPS, 2021. | Poison Suppression | |
| SCALE-UP | SCALE-UP: An Efficient Black-box Input-level Backdoor Detection via Analyzing Scaled Prediction Consistency. ICLR, 2023. | Input-level Backdoor Detection | black-box online detection |
| IBD-PSC | IBD-PSC: Input-level Backdoor Detection via Parameter-oriented Scaling Consistency. ICML, 2024. | Input-level Backdoor Detection | simple yet effective, safeguarded by theoretical analysis |
- DBD
- SS
- Neural Cleanse
- DP
- CutMix
- AEVA
The benchmark is coming soon.
| Organization | Contributors |
|---|---|
| Tsinghua University | Yiming Li, Mengxi Ya, Guanhao Gan, Kuofeng Gao, Xin Yan, Jia Xu, Tong Xu, Sheng Yang, Haoxiang Zhong, Linghui Zhu |
| Tencent Security Zhuque Lab | Yang Bai |
| ShanghaiTech University | Zhe Zhao |
| Harbin Institute of Technology, Shenzhen | Linshan Hou |
If our toolbox is useful for your research, please cite our paper(s) as follows:
@inproceedings{li2023backdoorbox,
title={{BackdoorBox}: A Python Toolbox for Backdoor Learning},
author={Li, Yiming and Ya, Mengxi and Bai, Yang and Jiang, Yong and Xia, Shu-Tao},
booktitle={ICLR Workshop},
year={2023}
}
@article{li2022backdoor,
title={Backdoor learning: A survey},
author={Li, Yiming and Jiang, Yong and Li, Zhifeng and Xia, Shu-Tao},
journal={IEEE Transactions on Neural Networks and Learning Systems},
year={2022}
}