Trying to backward through the graph a second time

[snykral]

RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.

This happens when calling loss_sampled.backward()

[Liuhong99]

I'm sorry this happened, but could you provide more context in your code so that we can know what's wrong with it? It's also helpful to refer to train_sophiag.py. That code should run without this issue.

[snykral]

I'm training a multimodal network. I've used optim.zero_grad(set_to_none=True) before updating the hessian, but it seems it had not detatched the grads from the network. I've designed an isolated function just to feed the model the same batch as the training step into the update hessian part. Sending the batch into a new function seems to reset the model grads. It's now working.