eigenvivek/DiffDRR

Gradient shenanigans with Siddon's method

Closed this issue · 3 comments

As far as I can tell, the gradients flowing through Siddon's method are broken for registration (from either #272 or #283).

In registration.ipynb, to get the image similarity metric to increase, I need to change maximize=False in the optimizer. That is, the gradients appear to be incorrectly pointing in the direction of steepest descent, for some mystical reason. Even after setting maximize=False, the image similarity metric maxes out at around 0.95 out of 1.

This hack isn't needed when using renderer="trilinear". The notebook works without issue with this backend (image similarity metric reaches 0.999 very quickly).

Siddon's method still works perfectly fine for reconstruction.ipynb, so I'm not sure what the issue is. Current best guess is that torch.nn.functional.grid_sample w/ mode="nearest" has broken something that previously worked with the custom implementation of nearest-neighbor interpolation?

Originally posted by @eigenvivek in #283 (comment)

Changing the plane offsets from 0.5 to 0 alleviates this issue but NO IDEA why 😵‍💫

alphax = torch.arange(dims[0] + 1).to(source) - 0.5
alphay = torch.arange(dims[1] + 1).to(source) - 0.5
alphaz = torch.arange(dims[2] + 1).to(source) - 0.5

Also found a bug: pretty sure alpha1 should be computed from dims + 1

alpha1 = ((dims - 1).to(source) - source) / sdd