Typo at page 37?

[ds2268]

When computing partial derivative of the SSE I think you made and error at indexing the sum. When you change output of the neuron from "z^(i)" to the SUM_i (w_j * x_j^(i)) I think it should be indexed through "j" not "i". As "z" is computed as the sum of products betweem weights and elements in x^(i).

[rasbt]

Thanks for the note. It definitely shouldn't be indexing over (i), which is already used as the sample index. j would also be wrong though because it's used for partial / partial w_j, i.e., the indexed parameter with respect to which we compute the derivative of z.

since

z = w_1 x_1 + ... w_n x_n = sum_{sth.} w_{sth} x_{sth}

we should select some index here that we don't use elsewhere, yet. E.g., k.

then, the derivative of "sum_k x_k w_k" with respect to "w_j" will be "x_j" if k=j, and 0 for j !=k so that we can remove the sum symbol.

Will add this to the errata. Thanks for pointing it out!!

[ds2268]

I agree I missed out the point that "j" is used for "w" and that there is one more "w_0" for for bias than there is features in "x" so we cannot use "j". Otherwise it might be ok to use "j" I guess.

[rasbt]

you are right regarding w_0, but just in case my previous comment was confusing, I meant the 0 independent from w_0 in that context. I.e., I meant that the derivative of a constant is zero. E.g., if we have a function

w_0 x_0 + w_1 x_1 + w_2 x_2 + ... + w_n x_n and then compute the partial derivative of the function with respect to e.g., w_1, we have "0 + x_1 + 0 + ... + 0