Contemporary-Neural-Network
This code implements a Classifier Neural Network using pure Scipy/Numpy (https://www.scipy.org/)
The code uses common contemporary algorithms for studying purposes, backprapogation is done with Tensor multiplication for better performance. The Iris Dataset is included for overfit testing.
Featuring:
ADAM optimizer Kingma, Diederik P., and Jimmy Ba. "Adam: A method for stochastic optimization." https://arxiv.org/abs/1412.6980
ELU activation function Clevert, Djork-Arné, Thomas Unterthiner, and Sepp Hochreiter. "Fast and accurate deep network learning by exponential linear units (elus)." https://arxiv.org/abs/1511.07289
HE initialization He, Kaiming, et al. "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification." Proceedings of the IEEE international conference on computer vision. https://www.cv-foundation.org/openaccess/content_iccv_2015/html/He_Delving_Deep_into_ICCV_2015_paper.html
Iris Species Dataset https://www.kaggle.com/uciml/iris
Backpropagation Dreyfus, Stuart (1962). "The numerical solution of variational problems". Journal of Mathematical Analysis and Applications. https://en.wikipedia.org/wiki/Backpropagation
Matthews Correlation Coefficient metric Matthews, Brian W. "Comparison of the predicted and observed secondary structure of T4 phage lysozyme." Biochimica et Biophysica Acta (BBA)-Protein Structure 405.2 (1975): 442-451. https://en.wikipedia.org/wiki/Matthews_correlation_coefficient