IntelLabs/AVUC

deterministic net has higher top 1 accuracy than bayesian net

RiqiangGao opened this issue · 2 comments

RiqiangGao commented

Dear authors,

Very impressive works.

I have a minor question. When I run the main_deterministic_cifar.py v.s. main_bayesian_cifar_avu.py, the deterministic net has a higher accuracy ( about ~4%) than the bayesian_cifar_avu. Is that normal?

thanks,

ranganathkrishnan commented

Hi @RiqiangGao, Thanks for the interest. You should be able to get comparable top-1 accuracy (within +/- 1% compared to deterministic model). May I know how many Monte Carlo samples are you using during prediction (num_monte_carlo)? Can you try the script 'scripts/test_bayesian_cifar_avu.sh', which will load our pretrained model.

If you are referring to the validation accuracy shown in the log during training, the accuracy will be lower as only 1 MC sample is used. But the test accuracy from "evaluate()" function will provide increased accuracy as multiple MC samples are drawn. I hope this helps!

-Ranganath

RiqiangGao commented

Thank you so much.

Indeed, I was referring to the validation accuracy shown in the log during training. Your response helped me to understand the code. Great work.

  • 👍1