Implementation of a customizable feed-forward neural network from scratch.
- Python 3.5.3 or newer.
- Numpy (for matrix operations & ndarrays)
- Scipy (for an accurate sigmoid function)
- [optional] Matplotlib (only for the plots in test.py)
Simply execute "test.py" to see it in action. "FFNN.py" contains the network itself.
- The 2D plot has fixed batch size and tests various learning rates
- The 3D plot tests out various batch sizes and learning rates at the same time. Comment out their respective sections to see their output.
The example neural net has 1 neuron on the input layer (receives x) and tries to generalize for the two equations (y1 and y2) for its output layer:
This implementation has following properties;
- Stochastic gradient descent, with modifiable mini-batch size.
- Shuffles training data after each epoch.
- Modifiable network structure.
- Can perform tests on a specified interval with the testing set and returns the cost (when a testing set is given).
- Weights and biases are randomly initialized.
- Customizable to contain other activation functions or cost functions (currently supporting sigmoid as the non-linear activation function and quadratic cost).
- Flexible to contain any type of input and output structure.
- Entire repo is GPL v3.