Implementation of Context-aware methods for multi-label classification to increase AP of minority co-occurrence pair.
Sogang Univ. Grad.
Python>=3.8
Pytorch>=1.10.0
pip install -r requirements.txt
├─augmentation.py # method1, method2, zero co-occurence pair 를 위한 함수
├─dataset.py
├─main.py
├─models.py
├─requirements.txt
├─utils.py
└─README.md
- Implementation of augmentation for minority co-occurrence pair(k-pair) from dataset
- Augmentation Method 2(patch-level; similar as Cutmix)
-
Train a model on MS-COCO 2017 train/val dataset
-
Test a trained model using MS-COCO 2017 test dataset
-
To train a model, we use Pascal-VOC 2012 trainval dataset
-
To test a model, we use Pascal-VOC 2007 test dataset
- We leverage Resnet-18, Resnet-50, Resnet-101 to test our augmentation method.
- Two major options for model, which are from scratch and from pre-trained weights on ImageNet.
- Our purpose of this augmentation is raising APs of minority class. It means that the model should be measured by classes-by-class AP tomeasure our augmentation method properly.
- If you want to apply the original setting, just use train.sh as a script
- To train model with another option, make your own script modifying the hyperparameters.
python3 main.py --lr 0.001 \
--batch-size 64 \
--scheduler cosine \
--criterion soft \
--device cuda:0 \
--use-method \
--rotate \
--save-dir runs/method
pip install gdown
gdown https://drive.google.com/uc?id=1wmSHTQnZbqFgaMIg-t-Ga3eocST5QVod # ResNet-101, K=30
class | mAP | #0 | #1 | #2 | #3 | #4 | #5 | #6 | #7 | #8 | #9 | #10 | #11 | #12 | #13 | #14 | #15 | #16 | #17 | #18 | #19 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
use-method | 86.24 | 91.96 | 94.7 | 90.65 | 88.31 | 66.1 | 84.85 | 95.14 | 96.6 | 74.97 | 75.66 | 81.75 | 96.6 | 95.97 | 92.54 | 97.57 | 72.41 | 65.46 | 79.93 | 99.45 | 84.24 |
no-method | 86.16 | 92.07 | 95.65 | 90. | 88.28 | 67.45 | 84.27 | 95.04 | 97.16 | 75. | 73.75 | 82.03 | 97.08 | 95.9 | 91.95 | 97.44 | 70.83 | 65.35 | 81.06 | 98.53 | 84.39 |
class | mAP |
---|---|
no-method | 76.46 |
use-method | 76.31 |
- To test properly, AP score matrix for multi-label classification
- MS-COCO dataloader with our augmentation method
- Apply K-select option with initial co-occurrence matrix
- Poisition randomness of attached object
- Size randomness of attached object
- Angle randomness of attached object
- Compare unrel_matrix and performances [evaluate heuristically]