Harryqu123/meta_confidence

[ECCV 2022] Improving the Reliability for Confidence Estimation

Improving the Reliability for Confidence Estimation

Introduction

This is an implementation of the method in Improving the Reliability for Confidence Estimation on MNIST and CIFAR-10.

If you find this code useful for your research, please consider citing:

@inproceedings{qu2022improving,
  title={Improving the reliability for confidence estimation},
  author={Qu, Haoxuan and Li, Yanchao and Foo, Lin Geng and Kuen, Jason and Gu, Jiuxiang and Liu, Jun},
  booktitle={European Conference on Computer Vision},
  pages={391--408},
  year={2022},
  organization={Springer}
}

Besides, this project is based on ConfidNet. Thus, you are also suggested to cite:

@article{corbiere2019addressing,
  title={Addressing failure prediction by learning model confidence},
  author={Corbi{\`e}re, Charles and Thome, Nicolas and Bar-Hen, Avner and Cord, Matthieu and P{\'e}rez, Patrick},
  journal={Advances in Neural Information Processing Systems},
  volume={32},
  year={2019}
}

Installation

1. Clone the repo.

2. Replace to original confidnet folder in ConfidNet with the confidnet folder in this repo.

3. Create a pretrained_models folder under the confidnet folder and put all stuffs in this link under folder pretrained_models.

4. Follow the installation instructions in ConfidNet.

Running the code

Execute the following command for training on MNIST:

./train_mnist_meta.sh

Execute the following command for training on CIFAR-10:

./train_cifar10_meta.sh

Acknowledgements

We thank the authors of ConfidNet for releasing the codes. Besides, we also thank the authors of the package learn2learn and the authors of Steep Slope Loss.