demirbilek95/Dynamics-of-Learning

The Dynamics of Learning Beyond Stochastic Gradient Descent

This repository contains the code used to obtain the experiment results of the thesis (the title is above). For the writing of the thesis, please check this repository. All of the experiments are stochastic. Setting a seed may influence the performance because the training phase takes benefit of this stochasticity. However, the results should be around the vicinity of the ones reported in the writings.

Structure of the Repository

• requirements.txt contains the modules that are used with their versions.
• plots contains the figures that are produced from the experiments.
• runs contains the accuracy and loss values from the various experiments.
• scripts includes the python files that are used from the notebooks.
  • architecture includes the neural network architectures that are used in the experiments.
  • data contains the code to produce the MNIST parity data and Random Data.
  • notebook_utils and plot_utils includes the helper functions that are used in notebooks.
  • optimizer includes the various optimizer algorithms that are used by the networks.
  • train and train_utils includes the reusable functions to train and test the networks.
• 01-Parity Experiments contains the code that are used to produce MNIST Parity Experiments.
• 02-Synthetic Data Experiments contains the code that are used to produce Synthetic Data Experiments.