Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial
(Code repository for the above titled paper)
The goal of the study is to compare several ML models and their uncertainty quantification capability for Engineering Design and Prognostics
Methods explored are:
- Gaussian Process Regression (GP): Rasmussen, Carl Edward, and Christopher KI Williams. "Gaussian processes in machine learning." Lecture notes in computer science 3176 (2004): 63-71.
- Neural Network Ensemble (NNE): Lakshminarayanan, Balaji, Alexander Pritzel, and Charles Blundell. "Simple and scalable predictive uncertainty estimation using deep ensembles." Advances in neural information processing systems 30 (2017).
- Monte Carlo Dropout (MC): Gal, Yarin, and Zoubin Ghahramani. "Dropout as a bayesian approximation: Representing model uncertainty in deep learning." international conference on machine learning. PMLR, 2016.
- Spectral-normalized Neural Gaussian Process (SNGP): Liu, Jeremiah, et al. "Simple and principled uncertainty estimation with deterministic deep learning via distance awareness." Advances in Neural Information Processing Systems 33 (2020): 7498-7512.
These methods are evaluated on two different case studies.
UQ methods are applied to two case studies
- Case study 1: Battery early life prediction
- Case study 2: Turbofan engine prognostics
All the models are built upon a ResNet-like architecture as shown below
The dataset for this case study was adopted from
- Severson, Kristen A., et al. "Data-driven prediction of battery cycle life before capacity degradation." Nature Energy 4.5 (2019): 383-391.
- Attia, Peter M., et al. "Closed-loop optimization of fast-charging protocols for batteries with machine learning." Nature 578.7795 (2020): 397-402.
The dataset can be summarized as follows
Dataset | No. of cells |
---|---|
Train | 41 |
Test1 | 43 |
Test2 | 40 |
Test3 | 45 |
See the directory for further details.
The dataset for this case study was adopted from
Contains code for the toy example mentioned in Section 3.5 of the paper.
The functional relationship between the output
The uncertainty maps of the UQ methods on this regression toy problem look as follows