Track gradients of activations in `HistogramLogger`
Opened this issue · 1 comments
callumm-graphcore commented
Is your feature request related to a problem? Please describe.
I don't think we track these at the moment
Describe the solution you'd like
We can do it using register_backward_hook
, presumably
Describe alternatives you've considered
We could just not do it and hope for the best idk
Additional context
No
callumm-graphcore commented
There might be complications arising from the fact that we take gradients of gradients during training (calculate gradients to compute the forces, then compare predicted forces to actual forces, then calculate gradients with respect to the forces loss)