google/nerfies

Jacobian computation

longbowzhang opened this issue · 0 comments

Hi,

I have a question w.r.t the computation of Jacobian matrix in warping.py.
For each input point, the network predicts the SE(3) transformation which is then used to warp the point to a new position.

In my opinion, the Jacobian of the new position w.r.t the input point position is essentially the rotation part of the SE(3). Therefore, since you already have the analytical Jacobian, I am wondering whether it is necessary to call Jax to compute it again as shown

out['jacobian'] = jac_fn(points, metadata_embed, extra)


Updated: I made a stupid mistake by ignoring the fact that the predicted SE(3) also depends on the input point. Thus, the Jacobian definitely is not equal to the rotation part of the SE(3).