The Pytorch implementation for the paper titled Wrapped Cauchy Distributed Angular Softmax for Long-Tailed Visual Recognition.
Visual recognition is vital for various computer vision applications. However, imbalanced or long-tailed data pose significant challenges to the deep learning approaches due to the mismatch between training and testing distributions. Our paper presents a novel softmax function based on wrapped Cauchy distribution: Wrapped Cauchy Distributed Angular Softmax (WCDAS). WCDAS considers the data-wise Gaussian-based kernels in the angular representation between features and classifier weights, describing noise and sparse sampling-induced uncertainty. As the class-wise distribution of such angular representation follows the sum of the kernels, we prove theoretically that the wrapped Cauchy distribution can be a better approximation for such mixed distributions than the widely-used Gaussian distribution. We demonstrate that WCDAS can dynamically optimize the compactness/margin of each class via the corresponding trainable concentration parameters. The empirical study shows that such class-wise parameters of WCDAS exhibit label-aware behavior. WCDAS outperforms other state-of-the-art softmax-based methods in long-tailed visual recognition on several benchmark datasets.
The algorithm is simply implemented in the class WCDAS at models/Loss.py.
To reproduce the results in the paper, the ResNet-10 equipped with the WCDAS is trained from scratch on ImageNet-LT dataset by
python main_train.py --dataset imagenetlt --net-config ResNet10Feature --workers 12 --seed 0 --loss-config WCDAS_ImageNetLT
python main_finetune.py --dataset imagenetlt --net-config ResNet10Feature_finetune --loss-config WCDAS_ImageNetLT --model-file ./results/imagenetlt_loss_WCDAS_ImageNetLT_ResNet10Feature_lr_0.4_model/ --workers 12 --seed 0 For iNaturalist-2018, the script to train from script is as follows:
python main_train.py --dataset 'inat2018' --net-config ResNet50Feature --workers 12 --seed 0 --loss-config WCDAS_iNaturalist2018
python main_finetune.py --dataset 'inat2018' --net-config ResNet50Feature_finetune --loss-config WCDAS_iNaturalist2018 --model-file ./results/inat2018_loss_WCDAS_iNaturalist2018_ResNet50Feature_lr_0.4_model/ --workers 12 --seed 0 | Method | ImageNet-LT | iNaturalist-2018 |
|---|---|---|
| WCDAS | 44.5 | 71.8 |
The codes are modified based on tvMF, Classifier-Balancing and BalancedMetaSoftmax-Classification.