samuela/git-re-basin

CIFAR-100 training logs

aktsonthalia opened this issue · 4 comments

aktsonthalia commented

Dear authors,

Amazing work and well-written paper!

I am trying to train a ResNet18 on CIFAR-100, and for my project I need to avoid using BatchNorm. As far as I understand, you used a ResNet20 model in your CIFAR-100 experiments, after having substituted BN with LN.

Did you also happen to train such a ResNet on the full CIFAR-100 dataset? If so: Is it possible for you to release your training logs or end results (the former is better, but the latter would already help me so much as a sanity check)?

Thanks a lot!
Ankit

samuela commented

Hi @aktsonthalia, thanks for your kind words! It's been a year or so since running these experiments so memory may be faulty but I don't recall completing a training run for a LayerNorm ResNet20 on the full CIFAR-100. IIRC we only used CIFAR-100 for the biased datasets experiments. If they were trained their logs should be in the weights&biases experiments!

aktsonthalia commented

Thanks for the quick response! I asked because I saw figures (Figure 5 and Figure 10) where the weight matching algorithm is compared with vanilla "Combined data training" (blue dotted line) on CIFAR-100. If I'm not wrong, that would be the experiment I'm looking for?

I could not find W&B links to these experiments in the repository. If you do come across them, it'd be great if you can share them with me! :) We can close this issue for now, though.

samuela commented

ah you're right! that training run is available here

  • 👍1
aktsonthalia commented

Massive help, thanks a lot!