VITA-Group/AugMax

I train ResNet18 in Cifar100

dreamer121121 opened this issue · 5 comments

dreamer121121 commented

I train ResNet18 in Cifar100 with augmax and without augmax but i find that i can got 77.5% accuracy without augmax ,while i can only got 76.3% with augmax, can u give some explainations?

htwang14 commented

Hi! What Lambda and steps values did you use? In the paper, we mentioned that on CIFAR100 with ResNet18 we use Lambda=1.0. steps should always be 10 on CIFAR10/100 and Tiny ImageNet. I've updated README in 8ca00c0 to reflect these details. Sorry for the confusion.

dreamer121121 commented

Hi! What Lambda and steps values did you use? In the paper, we mentioned that on CIFAR100 with ResNet18 we use Lambda=1.0. steps should always be 10 on CIFAR10/100 and Tiny ImageNet. I've updated README in 8ca00c0 to reflect these details. Sorry for the confusion.
Thanks for ur reply, i set the lambda=1.0 then i got ur results in CIfar100. but, i have another question, how to choose the lambda when using the augmax in different datasets?

  • 👍2
htwang14 commented

My suggestion is to first try Lambda~10. Usually AugMax gets similar performance with different Lambda values. The experiment on CIFAR100 with ResNet18 is the only exception I encountered, where smaller Lambda is preferred. My assumption is that ResNet18 has limited model capacity, so adding too much regularizations (by using a large Lambda) make the model over-smooth and hurts performance.

dreamer121121 commented

My suggestion is to first try Lambda~10. Usually AugMax gets similar performance with different Lambda values. The experiment on CIFAR100 with ResNet18 is the only exception I encountered, where smaller Lambda is preferred. My assumption is that ResNet18 has limited model capacity, so adding too much regularizations (by using a large Lambda) make the model over-smooth and hurts performance.

I want to apply Augmax to semantic segmentation, can you give me some advise?

htwang14 commented

I think AugMax is directly appliable on semantic segmentation tasks: just replace the classification loss with dense prediction loss (both the cross-entropy and the consistency loss should be replaced with a dense prediction version). As for benchmark datasets, you can use this one https://github.com/bethgelab/robust-detection-benchmark, which adds corruptions on segmentation/detection datasets. Although it's named as object detection, I think is can also be used for segmentation (at least Cityscapes and Pascal VOC have segmentation annotations).