hendrycks/robustness

Cifar10 Corrupted downloaded from tfds does not drop accuracy

saisubramaniam147 opened this issue · 1 comments

The dataset provided in https://www.tensorflow.org/datasets/catalog/cifar10_corrupted seems to have two variants:

  • Downloading the dataset directly without specifying the corruption e.g. tfds.load("cifar10_corrupted")
  • Downloading a specific corruption e.g. tfds.load("cifar10_corrupted/gaussian_noise_5")

The former variant does not seem to be affected by any corruption, for it does not lead to drop in the accuracy. Please confirm.

On a side note, I would also like to know more about how you calculated the classification error for Cifar datasets as reported in Table 1 of AugMix paper. Is it averaged over all the corruptions, or is there a separate Cifar10-C dataset that was created by combining all the corruptions at random?

I did not create cifar10_corrupted in TFDS, nor do I use tensorflow, so I do not know. Perhaps take the issue to them?