/energy_ood

Primary LanguagePythonApache License 2.0Apache-2.0

Energy-based Out-of-distribution Detection (Energy OOD)

This repository is the official implementation of Energy-based Out-of-distribution Detection by W. Liu, X. Wang, J. Owens, Y. Li. This method is an effective and easy OOD detector with and without fine-tuning. Our code is modified from Outlier-Exposure.

image

Pretrained Models and Datasets

Pretrained models are provided in folder

./CIFAR/snapshots/

Please download the datasets in folder

./data/

Testing and Fine-tuning

run energy score testing for cifar10 WRN

bash run.sh energy 0

run energy score testing for cifar100 WRN

bash run.sh energy 1

run energy score training and testing for cifar10 WRN

bash run.sh energy_ft 0

run energy score training and testing for cifar100 WRN

bash run.sh energy_ft 1

Results

Our model achieves the following average performance on 6 OOD datasets:

1. MSP vs energy score with and without fine-tuned on CIFAR-10

Model name FPR95
Softmax score 51.04%
Energy score (ours) 33.01%
Softmax score with fine-tune 8.53%
Energy score with fine-tune (ours) 3.32%

2. CIFAR-10 (in-distribution) vs SVHN (out-of-distribution) Score Distributions

image

3. Performance among different baselines for WideResNet

CIFAR-10:

Model name FPR95
Softmax score 51.04%
Energy score (ours) 33.01%
ODIN 35.71%
Mahalanobis 37.08%
Outlier Exposure 8.53%
Energy score with fine-tune (ours) 3.32%

CIFAR-100:

Model name FPR95
Softmax score 80.41%
Energy score (ours) 73.60%
ODIN 74.64%
Mahalanobis 54.64%
Outlier Exposure 58.10%
Energy score with fine-tune (ours) 47.55%

Outlier Datasets

These experiments make use of numerous outlier datasets. Links for less common datasets are as follows, 80 Million Tiny Images, Icons-50, Textures, Chars74K, and Places365, LSUN-C, LSUN-R, iSUN.

Citation

 @article{liu2020energy,
      title={Energy-based Out-of-distribution Detection},
      author={Liu, Weitang and Wang, Xiaoyun and Owens, John and Li, Yixuan},
      journal={Advances in Neural Information Processing Systems},
      year={2020}
 }