Releasing code for BA experiments?

[MarkYu98]

Hi, thank you for sharing this really nice work! The depth completion and visual odometry demos are really impressive.

I am very interested in the BA experiments in your paper, but I noticed the code is not included in this code repo. Do you plan to release the BA code later?

If there's no plan to release the BA code recently, do you mind explaining the subscripts σ_r^2I and K_c in equation (12) of the paper? I am a little confused by this notation. Are those subscripts actually typos and actually matrix multiplications?

Thanks!

[edexheim]

Hi Mark,

I wrote the bundle adjustment experiments using GTSAM, which would complicate the dependencies and installation of the project, so I focused on the components here and decided to keep this repository somewhat self-contained.

The subscripts refer to measurement covariances for a Mahalanobis distance. For example, those two components correspond to:

and

Let me know if you have more questions.

Eric

[MarkYu98]

Hi, thanks for your explanation! Do you mind explaining more about what exactly is (sigma_r)^2? Is it a hyperparameter as the homoscedastic observation noise variance (sigma_n)^2 mentioned? What's the difference between the subscript r and n?

Thanks again!

[edexheim]

$\sigma_r$ is the standard deviation you would expect for the reprojection error. You could find this empirically, but in practice I used 1 pixel which is somewhat standard. This accounts for sources of error such as keypoint detection which likely have sub-pixel errors.

$\sigma_n$ is the log-depth observation noise variance when learning the depth covariance function. When learning from a dataset, this noise accounts for independent measurement errors you may have from your depth sensor. At test time, if you have a model for how noisy log-depth observations are, you could also manually use these variances instead.

In summary, $\sigma_r^2$ is specifically the measurement error variance you expect for reprojection errors in bundle adjustment, while $\sigma_n^2$ is the variance for log-depth observations.

Let me know if that clears it up!
Eric

[MarkYu98]

Hi, thanks so much for your help! I have one more question regarding equation (12), should the depth in the second term be log-depth (i.e. log[T_{cW}P_c]z instead of [T{cW}P_c]_z)? Given that the mean m_c seems to represent the mean of log-depth elsewhere in the paper.

[edexheim]

Hi,

Yes, you are correct: that should be log-depth. Thank you for pointing this out, and I will correct the equation in arxiv.

Eric

[edexheim]

I have released the bundle adjustment example in this repository and the GTSAM factor implementation in a new repository https://github.com/edexheim/DepthCovFactors.

I am going to close this issue since the code is now released, but feel free to start a new one if there are more questions.