After loss.backward(), why needs torch.autograd.backward(inputs, grads, retain_graph=retain_graph)

Question

After loss.backward(), why needs torch.autograd.backward(inputs, grads, retain_graph=retain_graph)

Closed this issue 6 years ago · 1 comments

Hi Ratish, I have a problem here. After _compute_loss() and loss.div(normalization).backward(), why in function shards , there is torch.autograd.backward(inputs, grads, retain_graph=retain_graph) ? is it usual we use loss to do backward job? why use inputs backward again?

Answer 1 · 2019-04-24T10:33:13.000Z

Hi @LarryLee-BD, this repo is based upon a fork of OpenNMT-py. This thread OpenNMT/OpenNMT-py#387 gives the details of the two backward() operations.