3 Layer neural network built using python - Numpy Description of the project :
- We need to Import Numpy
- Declare Signoid function, Its a function that maps any value to value between 0 and 1 This is particularly usefull for creating probabilities out of numbers.
- Initialise input data. Lets say our input is ([[0,0,1] [0,1,1] [1,0,1] [1,1,1]]) It means we have 4 training examples with 3 neurons each.
- As we are generating random numbers, We seed them to make them deterministic.
- We define 2 layers of synopses for 3 layered neural network.
- Formulating training code: We create a for loop that iterates over our training code to optimisa the n/w for given data set.
- First layer is our input data.
- Prediction step involves performing matrix multiplication b/w each layer and its synopse. Then we run signoid function on all the values of our matrix to form our next layer.
- Next layer now contains prediction of our output data.
- We reapply the same process to to this layer now to get even more precise prediction
- Now that we have output value in layer2, we formulate error of layer2 as l2error=output-l2
- Print average error at a set interval to assure that it goes down every time.
- Multiply layer 2 error by our sigmoid function. This gives a derivative of output prediction from layer 2.
- This delta2 is used to reduce the error rate of our predictions when we update our synopses every iteration.
- Back Propogation: Then if we want to know how much layer 1 contributed to error in layer2 we multiply layer2Delta by synopses1's Transpose.
- Multiply layer 1 error by our sigmoid function. This gives a derivative of output prediction from layer 1.
- Gradient Descent: We use both these deltas to update the synopses
- l2 is the final output!!!