Significance of multiplying the translation vector by 0.1(args['scale'])

[rabin9948]

Hi,

Thank you for sharing the code.

I am not able to understand the significance of multiplying the translation vector by 0.1(args['scale']) constant in kitti.py file to update variables trajectory[i][0:3, 3].

Can you explain to me why you multiplied the translation vector by 0.1(args['scale'])?

        for i in range(len(trajectory)):
            trajectory[i] = np.dot(imu2cam, util.inv_SE3(trajectory[i]))
            trajectory[i][0:3, 3] *= self.args['scale']

[zachteed]

Hi, I did this so that the scales of the depth on KITTI, NYU and ScanNet matched approximately. On NYU the depth range was about 0.5-8 meters and KITTI is about 5-80 meters. The gt depth on KITTI is also scaled by a factor of 0.1.

[rabin9948]

Thank you for your reply!!

I am still trying to understand your code.

I have one more question.

Could you give me a little more information about _se3_matrix_expm_grad function in se3.py?

def _se3_matrix_expm_grad(op, grad):
    grad_upsilon_omega = tf.stack([
        grad[..., 0, 3],
        grad[..., 1, 3],
        grad[..., 2, 3],
        grad[..., 2, 1] - grad[..., 1, 2],
        grad[..., 0, 2] - grad[..., 2, 0],
        grad[..., 1, 0] - grad[..., 0, 1]
    ], axis=-1)

    return grad_upsilon_omega

[zachteed]

Hi, This is an approximation for the gradient of the se3_matrix_expm layer, it is exact when omega=0 and a very good approximation for small omega. It corresponds to taking the gradient of the taylor approximation

t ~= upsilon
R = I + [omega]_x

instead of the analytic formula. I was encountering numerical instabilities when trying to use automatic differentiation through the se3_matrix_expm function. Using this approximation fixed that issue without harming performance.

[rabin9948]

Thank you so much.
I thoroughly understand why you use the taylor approximation!

I have additional question.

transformation.py

 ### add dampening and apply increment ###
 H += ep_lmbda*tf.eye(6) + lm_lmbda*H*tf.eye(6)

Why do you add 'ep_lmbdatf.eye(6) + lm_lmbdaHtf.eye(6) ' to H?
And could you explain the 'ep_lmbdatf.eye(6) + lm_lmbdaHtf.eye(6) ' for me?

I am sorry to keep asking questions.

[zachteed]

Hi, the ep_lmbda and lm_lmbda add a small diagonal to the Hessian matrix H. This ensures that H is positive definite, so that the cholesky factorization will not fail. Additionally, it improves numerical stability by applying a small amount of damping to the pose update.

[rabin9948]

Thank you for your kindness!

I have two questions.

1. Why do you set 'ep_lmbda = 100'? ( I think the value of 'ep_lmbda' is too big for positive Hessian matrix H. )
2. Second question is related to _se3_matrix_expm_grad fuction. I guess that if omega is greater than the value of MIN_THETA, the result of _se3_matrix_expm_grad might incorrectly affect the gradient flow. Am I right?

[zachteed]

Hi, to answer your questions

1. The other values in the Hessian matrix are generally on the order of 1e6 or larger, so 100 is very small comparatively
2. _se3_matrix_expm_grad is an approximation of the gradient for small theta, so the accuracy of the gradient will degrade if theta becomes too large. However, this approximation is good for the data our method is trained on (i.e. we don't really expect the camera to rotate more than 20 degrees between pairs of frames).