ODIN: Out-of-Distribution Detector for Neural Networks

This is an unofficial PyTorch implementation for detecting out-of-distribution examples in neural networks. The method is described in the paper Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks by S. Liang, Yixuan Li and R. Srikant. The method reduces the false positive rate from the baseline 34.7% to 4.3% on the DenseNet (applied to CIFAR-10) when the true positive rate is 95%.

Experimental Results

We used two neural network architectures, DenseNet-BC and Wide ResNet. The PyTorch implementation of DenseNet-BC is provided by Andreas Veit and Brandon Amos. The PyTorch implementation of Wide ResNet is provided by Sergey Zagoruyko. The experimental results are shown as follows. The definition of each metric can be found in the paper.

Running the code

Dependencies

Install requirements:

while read requirement; conda install --yes $requirement;or pip install $requirement; end < requirements.txt

Download datasets:

cd data
chmod 777 ./download_data.sh
./download_data.sh

Running

Here is an example code reproducing the results of DenseNet-BC trained on CIFAR-10 where TinyImageNet (crop) is the out-of-distribution dataset. The temperature is set as 1000, and perturbation magnitude is set as 0.0014. In the root directory, run

cd src
python main.py --nn densenet10 --dataset Uniform --magnitude 0.0014 --temperature 1000

or using tmux

cd bash_scripts
tmuxp load produce_score.yaml

Analyze

cd notebooks
jupyter lab

License

Please refer to the LICENSE.