Question about camera pose optimization in `cameras.py`

Question

Question about camera pose optimization in `cameras.py`

Closed this issue 10 days ago · 3 comments

Lee-JaeWon commented 2 months ago

Thanks for the good work.

I noticed that there is a commented camera pose optimization code in cameras.py, does it work?

I'm going to try to make it work, but how can I make sure it's optimized well?

Thanks in advance for your kind response.

Answer 1 · 2024-08-12T07:46:20.000Z

Hi, I have implemented the gradients w.r.t. camera extrinsics in cuda. However, this has not yet been fully tested, I am not sure about its correctness. You may check or modify it in you own experiments.

Answer 2 · 2024-08-12T12:14:36.000Z

@turandai
I've tried it and don't see any significant impact.

Is there anything else I should consider, and in what situations have you had success?

Answer 3 · 2024-08-14T11:51:01.000Z

I tried on DTU, with random noise added to the GT poses. The results are better with pose optimization than without, but still worse than using the GT poses. I am not sure about the problem but I think maybe the per-image sampling strategy of the 3dgs rasterizer is not good for pose optimization? You might check other works focusing on pose optimization for more insights.