deeplearning-wisc/Spurious_OOD

Question About Spurious OOD Dataset Generation for WaterBirds

Closed this issue · 4 comments

dubowsky commented

Hi,

Thanks for your nice work in pointing out the impact of spurious effect on the OOD. I have a question about the generation of OOD dataset for WaterBirds.

The code seems that the OOD dataset is directly the Place365 dataset instead of the example shown in the figure 1 (Right), a boat in the lake. I think the shown example in the paper indicates that the Spurious OOD figure should contain an OOD object inside. Is my understanding correct? Feel free to correct me if there is any problem. Thanks!

alvinmingsf commented

Thanks for raising this issue. Yes, as also mentioned in readme, generate_placebg.py is used to generate background images. Feel free to place different OOD objects of your interest on top of background images and see how the detection performance is influenced. In our initial experiments, we tried several objects but it turns out they do out have much impact on the performance especially when the network relies on background spurious correlation during training.

dubowsky commented

Thanks for your reply and it's clear. One follow-up question is what kinds of the 'topping' objects/datasets you used in the trials? I would like to try with different 'toppings' as well.

alvinmingsf commented

Thanks for the question! By the definition of OOD, objects with labels other than the ID labels are considered valid OOD. For Waterbirds, objects like "boat", "cat" or "dogs" are all suitable toppings.

dubowsky commented

Clear. Thanks for your reply!