A simple neural network and optimization package implemented from scratch in Python using no libraries other than NumPy.
/optimization contains utlity functions and algorithms for gradient-based function optimization.
/neural_network contains a multilayer neural network implementation, including a flexible implementation of the backpropogation algorithm.
The neural network implementation was trained on the MNIST handwritten digits database and achieved a test set accuracy of over 97.5%.
In addition to code, the repo contains detailed derivations of all loss function gradients used in the project, including a derivation of the backpropagation algorithm. These derivations can be found in the iPython notebooks /optimization/optimization.ipynb and /neural_network/neural_net.ipynb.