SaliencyMix [1] with more randomness.
Recently the size of the vision models has been rapidly increasing, resulting powerful representation ability of the model. However, the dataset size needed to train such a model without overfitting is also growing fast. But obtaining more data is often difficult, limiting the practical applicability of the large vision models. Data augmentation is one of the most efficient strategies to alleviate the issue. CutMix [2] randomly replaces a random image region with a random patch from another training image. SaliencyMix [1] further improves this idea using a saliency map, assuring selected random patch contains relevant information about the source object. However, SaliencyMix [1] fixes the location of the patch being cropped and the location of the region being replaced. We think its deterministic “crop-and-paste” strategy limits the full potential of SaliencyMix [1] because of the lack of diversity of training data. Therefore, we aim to find a better “pasting” strategy that cares about the region being replaced. We will compare our strategy with the random pasting strategy of [1] and [2].
Improving the data augmentation technique of the target paper [1] by adding more randomized behavior.
- Implement random location cropping/pasting proportional to saliency intensity.
- Implement a label mixing algorithm considering not only patch size but also saliency intensity.
- Implement a hybrid method of CutMix [2] and MixUp [3], which can further increase the diversity of the training dataset.
[1] A F M Shahab Uddin and Mst. Sirazam Monira and Wheemyung Shin and TaeChoong Chung and Sung-Ho Bae. “SaliencyMix: A Saliency Guided Data Augmentation Strategy for Better Regularization.” International Conference on Learning Representations. 2021.
[2] Sangdoo Yun and Dongyoon Han and Seong Joon Oh and Sanghyuk Chun and Junsuk Choe and Youngjoon Yoo. “CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features.” Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019.
[3] Hongyi Zhang and Moustapha Cisse and Yann N. Dauphin and David Lopez-Paz. “Mixup: Beyond Empirical Risk Minimization.” International Conference on Learning Representations. 2018.
conda create -n rand_saliency_mix python=3.10
conda activate rand_saliency_mix
pip install -r requirements.txtTo train Resnet with RandSaliencyMix:
python codes/train.py --config configs/train/res50_tinyimnet_randsalmix.yaml --gpu 0,1