/dropout-in-rnn

This is my master thesis project. The goal is to apply dropout within RNN in order to obtain uncertainty for predictions.

Primary LanguageJupyter Notebook

Dropout as Bayesian Approximation in RNN

Master thesis - Aalto University - Machine Learning and Data Science

Dataset

TODOs

  • Add biases
  • Make loss function
  • Stack multiple LSTM layers
  • Handle single dimension input
  • Iterate Stochastic pass once during training
  • Use parameter optimizer instead of writing custom loss functions
  • Make stochastic modules for regression
  • Make stochastic modules for classification
  • Compute predictive variance for classification
  • Compute predictive variance for regression
  • Optimize dropout rate (using Concrete Dropout)
  • Download clinical data
  • Early stop in training (use validation error)
  • Make equivalent in Tensorflow