The same experiment was run twice and the results were very different
Closed this issue · 3 comments
My setting:
Fine-tuning on CIFAR 10 using ImageNet pre-trained weights.
First time:
1st epoch Acc. 90.24
Second time:
1st epoch Acc. 10.79
Does this happen often?
Thanks for your great work and now I'm exploring the details. Looking forward to your response.
Hello @YutingLi0606 , Something should be wrong for your second run. Results should be stable.
Best
Thanks for the reminder. I've evaluated your architecture by some toy experiments, and using KAT improved accuracy by 1-2% on CIFAR-10 but decreased it by 2% on CIFAR-100. Are you considering including experiments on small-scale datasets to broaden the impact of your work?
Thank you so much for the new experiments! Can you share you experimental details (Maybe send me a email)? I can take a look and re-run your experiments, and see if there is any method to further improve the results.
Best,