NeurIPS 2022 Accepted Paper - Quantization, Efficient Inference, Imbalance Learning (Long-tail)
-
Motivation: to diminish the quantization error from the heterogeneity of class distributions (variations & sample numbers)
-
Method: class distribution scaling + rebalance loss for homogeneity of class variances
- python3
- pytorch==1.7.1
- cudatoolkit==11.0.221
- numpy==1.19.2
- tensorboardx==1.4
- e.g. 4-bit ResNet-20 on CIFAR-10-LT (gamma = 10).
- e.g. CIFAR-10.
cd src/
mkdir data_dir/
Save training (testing) image data to the path src/data_dir/cifar10/. (e.g. src/data_dir/cifar10/train/airplane/0001.jpg)
- e.g. CIFAR-10-LT (gamma = 10).
python3 longTailDataGeneator.py --data_dir "src/data_dir/cifar10/" --output_csv "cifar10.csv" --gamma 10
Files 'cifar10_train10.csv' and 'cifar10_test.csv' are shown under src/.
mv -r *.csv resnet-20-cifar-10/data/
ps. These two steps are neglibible since the csv files have been generated in the specific directories (see the next step).
- e.g. 4-bit ResNet-20 on CIFAR-10-LT (gamma = 10).
cd resnet-20-cifar-10/
Generated csv files are located under data/.
Pretrain and save models to the path resnet-20-cifar-10/pretrained/.
python3 main.py --csv_dir data/ --job_dir "experiment/ours/resnet/t_4bit_pre32bit" --method "ours" --source_dir "pretrained" --source_file "res20_32bit/model_best.pt" --arch resnet --bitW 4 --abitW 4 --target_model "resnet20_quant" --source_model "resnet20" --lt_gamma 10 --num_epochs 50 --train_batch_size 128 --eval_batch_size 100 --lr 0.04
@article{chen2022climbq,
title={ClimbQ: Class Imbalanced Quantization Enabling Robustness on Efficient Inferences},
author={Chen, Ting-An and Yang, De-Nian and Chen, Ming-syan},
journal={Advances in Neural Information Processing Systems},
volume={35},
pages={37134--37145},
year={2022}
}