/Adversarial-AutoMixup

Official PyTorch(MMCV) implementation of “Adversarial AutoMixup” (ICLR 2024 spotlight)

Primary LanguagePythonApache License 2.0Apache-2.0

Adversarial AutoMixup (ICLR 2024 spotlight)

Huafeng Qin1,*,†, Xin Jin1,*, Yun Jiang1, Mounim A. El-Yacoubi2, Xinbo Gao3

1Chongqing Technology and Business University

2Telecom SudParis, Institut Polytechnique de Paris

3Chongqing University of Posts and Telecommunications

* Equal Contribution † Corresponding Author

We propose AdAutoMix, an adversarial automatic mixup augmentation approach that generates challenging samples to train a robust classifier for image classification, by alternatively optimizing the classifier and the mixup sample generator. AdAutoMix comprises two modules, a mixed example generator, and a target classifier. The mixed sample generator aims to produce hard mixed examples to challenge the target classifier while the target classifier’s aim is to learn robust features from hard mixed examples to improve generalization. To prevent the collapse of the inherent meanings of images, we further introduce an exponential moving average (EMA) teacher and cosine similarity to train AdAutoMix in an end-to-end way.

Mixed Images of Various Mixup-based Approaches.

📬 You can contact me by email: 158398730@qq.com or WeChat: xinxinxinxin_j.

If you are interested in palm or finger vein research, please contact us!

🛠 Installation

💥News! ! !💥
2024-01-18: you can clone Openmixup training "AdAutoMix"[ICLR 2024].
2023-12-19: We update some analysis tools code such as: Calibration, FGSM calibration_fgsm.py and Occlusion Robustness occlusion_robustness.py experiments, also we support a mix augmentation method "SnapMix"[AAAI 2021].
Big thanks to Siyuan Li(@Lupin1998).

🔧How to install?🔧
In fact, you can add our python file in OpenMixup.
There, you can see how to use it and the environment required. What you need to do is add or replace our files by folder inside OpenMixup, and then add the function names of the files in the __init__.py file.
You also can download or find other Mixup methods in OpenMixup("https://github.com/Westlake-AI/openmixup")
Thanks contributors: Siyuan Li(@Lupin1998), Zichen Liu(@pon7) and Zedong Wang(@Jacky1128).

Here are the commands to install OpenMixup

conda create -n openmixup python=3.8 pytorch=1.12 cudatoolkit=11.3 torchvision -c pytorch -y
conda activate openmixup
pip install openmim
mim install mmcv-full
git clone https://github.com/Westlake-AI/openmixup.git
cd openmixup
python setup.py develop

Here are the commands to git clone AdAutoMixup

git clone https://github.com/JinXins/Adversarial-AutoMixup.git

📊 Experiments

CIFAR-100

Name alpha Conference ResNet18 ResNeXt50 Swin-Tiny ConvNeXt-T
Vanilla - 78.04 81.09 78.41 78.70
MixUp 1.0 ICLR2018 79.12 82.10 76.78 81.13
CutMix 0.2 ICCV2019 78.17 78.32 80.64 82.46
SaliencyMix 0.2 ICLR2021 79.12 78.77 80.40 82.82
FMix 0.2 ArXiv 79.69 79.02 80.72 81.79
ResizeMix 1.0 CVMJ2023 80.01 80.35 80.16 82.53
PuzzleMix 1.0 ICML2020 81.13 82.85 80.33 82.29
AutoMix 2.0 ECCV2022 82.04 83.64 82.67 83.30
AdAutoMix 1.0 ICLR2024 82.32 84.22 84.33 83.54

Tiny-ImageNet & ImageNet-1K(denote *)

Name alpha Conference ResNet18 ResNeXt50 ResNet18* ResNet34* ResNet50*
Vanilla - 61.68 65.04 70.04 73.85 76.83
MixUp 1.0 ICLR2018 63.86 66.36 69.98 73.97 77.12
CutMix 0.2 ICCV2019 65.53 66.47 68.95 73.58 77.17
SaliencyMix 0.2 ICLR2021 64.40 66.55 69.16 73.56 77.14
FMix 0.2 ArXiv 63.47 65.08 69.96 74.08 77.19
ResizeMix 1.0 CVMJ2023 63.17 65.87 69.50 73.88 77.42
PuzzleMix 1.0 ICML2020 65.81 67.83 70.12 74.26 77.54
AutoMix 2.0 ECCV2022 67.33 70.72 70.50 74.52 77.91
AdAutoMix 1.0 ICLR2024 69.19 72.89 70.86 74.82 78.04

CUB-200, FGVC-Aircraft and Standford Cars

Name alpha Confrence CUB R18 CUB R50 FGVC R18 FGVC RX50 Cars R18 Cars RX50
Vanilla - 77.68 82.38 80.23 85.1 86.32 90.15
MixUp 1.0 ICLR2018 78.39 82.98 79.52 85.18 86.27 90.81
CutMix 0.2 ICCV2019 78.40 83.17 78.84 84.55 87.48 91.22
ManifoldMixup 2.0 ICML2019 79.76 83.76 80.68 86.6 85.88 90.20
SaliencyMix 0.2 ICLR2021 77.95 82.02 80.02 84.31 86.48 90.60
FMix 0.2 ArXiv 77.28 83.34 79.36 86.23 87.55 90.90
ResizeMix 1.0 CVMJ2023 78.5 83.41 78.1 84.08 88.17 91.36
PuzzleMix 1.0 ICML2020 78.63 83.83 80.76 86.23 87.78 91.29
AutoMix 2.0 ECCV2022 79.87 83.88 81.37 86.72 88.89 91.38
AdAutoMix 1.0 ICLR2024 80.88 84.57 81.73 87.16 89.19 91.59

If you want see more results, please check this Experiments.md.

😉 Citation

If you feel that our work has contributed to your research, please cite it, 🥰 and please don`t forget to cite OpenMixup if you use this project ! 🤗 Thanks.

@inproceedings{iclr2024adautomix,
      title={Adversarial AutoMixup},
      author={Huafeng Qin and Xin Jin and Yun Jiang and Mounim A. El-Yacoubi and Xinbo Gao},
      booktitle={International Conference on Learning Representations},
      year={2024},
}

@article{li2022openmixup,
  title = {OpenMixup: A Comprehensive Mixup Benchmark for Visual Classification},
  author = {Siyuan Li and Zedong Wang and Zicheng Liu and Di Wu and Cheng Tan and Stan Z. Li},
  journal = {ArXiv},
  year = {2022},
  volume = {abs/2209.04851}
}

📝 Interest

  If you are interested in vein research (palm veins, finger veins, etc.), why not contact us and we will be happy to discuss the research or questions with you.
  Of course, we're also trying to research how we can use AI to predict and detect diseases, potentially Alzheimer's, Parkinson's, etc., but of course, this is new research for us, so we're still figuring it out.
  Email: 158398730@qq.com