the normalization of OOD test datasets

[Esther-PAN]

Hello, I have encountered an issue regarding the normalization of OOD test datasets. As per my understanding, during the evaluation of OOD datasets, it is necessary to perform normalization specific to each OOD test dataset. Taking CIFAR-100 as an example, where can I find the mean and standard deviation for the normalization of various OOD datasets? Thank you

[Esther-PAN]

for example， the mean and std for svhn\tin\texture\places365....

[zjysteven]

Actually one should NOT "perform normalization specific to each OOD test dataset". By doing so you are assuming that you know which dataset the input is from in the first place, which makes OOD detection meaningless (you already know whether the input is OOD or ID).

Therefore we just use the normalization parameters corresponding to each ID dataset, which we believe is reasonable and also practical. You can find the parameters here:

OpenOOD/openood/preprocessors/transform.py

Lines 3 to 12 in 555d879

	normalization_dict = {
	'cifar10': [[0.4914, 0.4822, 0.4465], [0.2470, 0.2435, 0.2616]],
	'cifar100': [[0.5071, 0.4867, 0.4408], [0.2675, 0.2565, 0.2761]],
	'imagenet': [[0.485, 0.456, 0.406], [0.229, 0.224, 0.225]],
	'imagenet200': [[0.485, 0.456, 0.406], [0.229, 0.224, 0.225]],
	'covid': [[0.4907, 0.4907, 0.4907], [0.2697, 0.2697, 0.2697]],
	'aircraft': [[0.5, 0.5, 0.5], [0.5, 0.5, 0.5]],
	'cub': [[0.5, 0.5, 0.5], [0.5, 0.5, 0.5]],
	'cars': [[0.5, 0.5, 0.5], [0.5, 0.5, 0.5]],
	}

[Esther-PAN]

Thank you for your prompt reply！
Can I understand it this way: if I use CIFAR-100 as the ID dataset, then when conducting out-of-distribution detection on various OOD datasets, all OOD datasets should be normalized using the same normalization parameters as CIFAR-100?

[Esther-PAN]

Or OOD test datasets do not need to be normalized to maintain the differences in their distributions?

[zjysteven]

if I use CIFAR-100 as the ID dataset, then when conducting out-of-distribution detection on various OOD datasets, all OOD datasets should be normalized using the same normalization parameters as CIFAR-100

Correct. You can also imagine that in practice, a deployed image classifier typically uses a fixed set of preprocessing operations.

[Esther-PAN]

thanks sooo much！