Replace log_alpha and log_labda in optimizers to alpha
Closed this issue · 1 comments
rickstaa commented
We have to make sure that we are not introducing NaN bugs due to the log we use in the optimizer. This can be done by replacing it with the exponential value or clipping the log. For more information on this kind of problems see this talk. The tf.debugging.check_numerics method can be used if such errors are present in the code. It can be enabled by adding code to your file:
tf.debugging.enable_check_numerics()This can easily be solved using a property that clips the log_lambda to be bigger than -80 (about the min precision of 32-bit variable).