AdaptiveSoftmaxCrossEntropy crashes by ZeroDivisionError
butsugiri opened this issue · 2 comments
butsugiri commented
AdaptiveSoftmaxCrossEntropy crashes by ZeroDivisionError in forward_shared_part .
I suspect that it occurs when every element of in_cluster in line179 is False (i.e. every index of the sample belong to the "head" part of output layer), and reduced_x gets the value 0 in axis-0 after the linear layer is calculated.
The log is:
File "/home/kiyono/.pyenv/versions/anaconda3-4.4.0/envs/nc/lib/python3.6/site-packages/chainer/
training/updater.py", line 223, in update
self.update_core()
File "/home/kiyono/.pyenv/versions/anaconda3-4.4.0/envs/nc/lib/python3.6/site-packages/chainer/
training/updater.py", line 236, in update_core
optimizer.update(loss_func, **in_arrays)
File "/home/kiyono/.pyenv/versions/anaconda3-4.4.0/envs/nc/lib/python3.6/site-packages/chainer/
optimizer.py", line 528, in update
loss = lossfun(*args, **kwds)
File "/home/kiyono/deploy/rec/bin/EncoderDecoder/net.py", line 571, in __call__
dec_loss = self.output.output_and_loss(last_fc_input, cat_dec_target) * cat_dec_target.shape[0] / ba
tch_size
File "/home/kiyono/deploy/rec/bin/EncoderDecoder/adaptive_softmax.py", line 459, in outpu
t_and_loss
h, t, Ws, Rs, cutoff, normalize=False, reduce='mean')
File "/home/kiyono/deploy/rec/bin/EncoderDecoder/adaptive_softmax.py", line 414, in adapt
ive_softmax_cross_entropy
x, t, *Ws, *Rs)
File "/home/kiyono/.pyenv/versions/anaconda3-4.4.0/envs/nc/lib/python3.6/site-packages/chainer/
function.py", line 212, in __call__
ret = node.apply(inputs)
File "/home/kiyono/.pyenv/versions/anaconda3-4.4.0/envs/nc/lib/python3.6/site-packages/chainer/
function_node.py", line 221, in apply
outputs = self.forward(in_data)
File "/home/kiyono/.pyenv/versions/anaconda3-4.4.0/envs/nc/lib/python3.6/site-packages/chainer/
function.py", line 117, in forward
return self._function.forward(inputs)
File "/home/kiyono/.pyenv/versions/anaconda3-4.4.0/envs/nc/lib/python3.6/site-packages/chainer/
function.py", line 319, in forward
return self.forward_gpu(inputs)
File "/home/kiyono/deploy/rec/bin/EncoderDecoder/adaptive_softmax.py", line 236, in forwa
rd_gpu
log_y = self.forward_shared_part(inputs)
File "/home/kiyono/deploy/rec/bin/EncoderDecoder/adaptive_softmax.py", line 185, in forwa
rd_shared_part
ls_out = log_softmax._log_softmax(out)
File "/home/kiyono/.pyenv/versions/anaconda3-4.4.0/envs/nc/lib/python3.6/site-packages/chainer/
functions/activation/log_softmax.py", line 36, in _log_softmax
x.reshape(x.shape[:2] + (-1, 1)))
File "cupy/core/core.pyx", line 482, in cupy.core.core.ndarray.reshape
File "cupy/core/core.pyx", line 455, in cupy.core.core.ndarray._reshape
File "cupy/core/internal.pyx", line 139, in cupy.core.internal.infer_unknown_dimension
ZeroDivisionError: integer division or modulo by zero