/dice-neurips2022

Code for "On the Effectiveness of Sparsification for Detecting the Deep Unknowns"

Primary LanguagePython

Less is More: Leveraging Sparse Weights for Detecting the Deep Unknowns

In this work, we reveal important insights that reliance on unimportant weights and units can directly attribute to the brittleness of OOD detection. To mitigate the issue, we propose a sparsification-based OOD detection framework termed DICE. Our key idea is to rank weights based on a measure of contribution, and selectively use the most salient weights to derive the output for OOD detection

Usage

1. Dataset Preparation for Large-scale Experiment

In-distribution dataset

Please download ImageNet-1k and place the training data and validation data in ./datasets/ILSVRC-2012/train and ./datasets/ILSVRC-2012/val, respectively.

Out-of-distribution dataset

We have curated 4 OOD datasets from iNaturalist, SUN, Places, and Textures, and de-duplicated concepts overlapped with ImageNet-1k.

For iNaturalist, SUN, and Places, we have sampled 10,000 images from the selected concepts for each dataset, which can be download via:

./download_ood.sh

For Textures, we use the entire dataset, which can be downloaded from their original website.

Please put all downloaded OOD datasets into ./datasets/.

Pre-trained model

Please download Bit-S-R101x1 and place in the checkpoints folder.

2. Dataset Preparation for CIFAR Experiment

In-distribution dataset

The downloading process will start immediately upon running.

Out-of-distribution dataset

We provide links and instructions to download each dataset:

  • SVHN: download it and place it in the folder of datasets/ood_datasets/svhn. Then run python select_svhn_data.py to generate test subset.
  • Textures: download it and place it in the folder of datasets/ood_datasets/dtd.
  • Places365: download it and place it in the folder of datasets/ood_datasets/places365/test_subset. We randomly sample 10,000 images from the original test dataset.
  • LSUN-C: download it and place it in the folder of datasets/ood_datasets/LSUN.
  • LSUN-R: download it and place it in the folder of datasets/ood_datasets/LSUN_resize.
  • iSUN: download it and place it in the folder of datasets/ood_datasets/iSUN.

For example, run the following commands in the root directory to download LSUN-C:

cd datasets/ood_datasets
wget https://www.dropbox.com/s/fhtsw1m3qxlwj6h/LSUN.tar.gz
tar -xvzf LSUN.tar.gz

Preliminaries

It is tested under Ubuntu Linux 20.04 and Python 3.8 environment, and requries some packages to be installed:

Demo

1. Demo code for Large-scale Experiment

Run ./demo-imagenet.sh.

2. Demo code for CIFAR Experiment

Run ./demo-cifar.sh.