about the input range

[ziaoang]

in the tutorial.lua, you remove the mean of items:
'''
train[k][{ {},2 }]:add(-mean)
if test[k] then test[k] [{ {},2 }]:add(-mean) end
'''
but this will break the input range [-1, 1].

[fstrub95]

Hi,
You are right, centering the input can break the input range.
Actually, It is not an issue to break the range for input data. Yet, breaking the target range (input=target in autoencoders) may be troublesome since the target becomes unreachable for Tanh() transfer function.
The best way would be to recenter the target on every step to avoid this saturation.

However, I have not observed an impact by slightly breaking this range in practice. On the other side, if the mean is not removed, the network fail to learn the bias parameters and the final score is far lower.

So why using this piece of code. Well, updating the input/target once for all at the beginning make the code faster and easier to read. (which is crucial for a tutorial!)

Kind regards,
STRUB Florian

[ziaoang]

Thank you for your reply, and congratulations that your paper has been accepted:)