/SWA-Gaussian-TF2

TensorFlow implementation of "A Simple Baseline for Bayesian Uncertainty in Deep Learning"

Primary LanguagePythonMIT LicenseMIT

[TensorFlow 2] A Simple Baseline for Bayesian Uncertainty in Deep Learning: SWA-Gaussian (SWAG)

TensorFlow implementation of "A Simple Baseline for Bayesian Uncertainty in Deep Learning"

Concept

Algorithm to utilize the SWAG [1].

Equation for the weight sampling from SWAG [1].

Results

The red color and the blue color represent the initial state and current state respectively.

Variable MNIST CIFAR10

Performance

MNIST

Method Accuracy Precision Recall F1-Score
Final Epoch 0.99230 0.99231 0.99222 0.99226
Best Loss 0.99350 0.99350 0.99338 0.99344
SWAG (S = 30) 0.99310 0.99305 0.99299 0.99302
SWAG (Last Momentum) 0.99340 0.99340 0.99330 0.99335

CIFAR10

Method Accuracy Precision Recall F1-Score
Final Epoch 0.73130 0.73349 0.73130 0.73147
Best Loss 0.73240 0.73205 0.73240 0.73099
SWAG (S = 30) 0.74100 0.74622 0.74100 0.74260
SWAG (Last Momentum) 0.73490 0.73888 0.73490 0.73561

Requirements

  • Python 3.7.6
  • Tensorflow 2.3.0
  • Numpy 1.18.15
  • whiteboxlayer 0.1.15

Reference

[1] Wesley Maddox et al. (2019). A Simple Baseline for Bayesian Uncertainty in Deep Learning. arXiv preprint arXiv:1902.02476.