Not getting desired accuracy on CIFAR-100

[Puneet2000]

I am getting around 75% accuracy on ResNet34 on CIFAR100. Hyperparameters are default.

[TimDettmers]

Thanks for opening an issue on this. From the PyTorch CIFAR-100 repo I find that the performance should be around 78%. Default parameters means 5% weights which is probably not enough to replicate dense performance. From my experience with ResNets you will need about 30% weights to replicate dense performance. The learning rate schedule might also be a bit off. If you want to get better performance, you should also try to tune the learning rate. The training curve differs slightly between sparse and dense networks.

[Puneet2000]

Thanks, @TimDettmers!