SethHWeidman/ODSC_Neural_Nets_11-04-17

2D matrix multiplication computational graph in "Neural networks from Scratch"

Opened this issue · 0 comments

Good Evening Seth Weidman.
Firstly, I appreciate the wonderful work you are doing to teach neural networks. I am reading your book, "Neural networks from Scratch". Things have been moving so far well until i reached the 2D matrix multiplication computational graph. I read from some book and it states that the derivative of a matrix with respect to a matrix is a tensor, how comes that we differentiate a 3x2 matrix wrt a 3x2 matrix and get back a 3x2 mat not a (3x2x3x2) tensor? I am really enjoying this book, I had skipped this part unfortunately I can't understand all the following chapters minus a solid understanding of this.
This is the eqn that we have in the book
image
but i do not understand why we are finding this yet it is not in the equation
image
I will be grateful incase you reply, we are having a final graduation project and your book is our savior, we are now at pause since we can not proceed any further without understanding what is taking place here. Thank you