This notebook implements a 2 layer MLP from scratch, without using any autodiff libraries. Various methods of optimisation are then implemented and applied to optimise instances of the network, with a grid search used to find optimial hyperparamters based on performance on a validation data set. Although not the most efficient implementation, this notebook is a simple demonstration with primary purpose to further my understanding on the mathematics & theory of deep learning.
The code itself is written in Julia, with a hack by Pontus Stenetorp (https://pontus.stenetorp.se/) allowing for Julia support in google colab notebooks.