NVlabs/intrinsic3d

Bug report: image project out-of-range

dolphin-li opened this issue · 4 comments

sdf/colorization.cpp
line 228~231:
if (valid) valid = cam_.project(pt_tf, pt2f, pt2i); if (valid) valid = isVoxelVisible(pt_tf, depth, pt2i[0], pt2i[1]);
pt2i may be outside the depth image, and the valid check has no effect. The reason is that cam_.width() and cam_.height() are on the finest image pyramid level, while depth may be at corser level.

This causes crash on the "lion" data.
Hope it would be helpful.
Thanks.

Maybe we can narrow it down a bit:

  • The depth map is always resized to the size of the color image first, as done in Intrinsic3D::init() (refinement/intrinsic3d.cpp, line 182). Hence intrinsics for depth and color are the same (i.e. the color intrinsics) from this point on.
  • Both color and depth are downsampled in refinement/intrinsic3d.cpp, line 184 then, meaning that depth and color are always accessed at the same resolution/level.
  • In Camera::project(const Vec3f &pt, Vec2f &pt2f, Vec2i &pt2i) const in camera.cpp, it is checked that pt2i is within width and height, returning false otherwise.
    Hence I think that I'm checking for this, or can you maybe point me to something I'm missing?

Just to make sure: does the crash maybe have any other reason? I.e. not enough RAM? You shuld have 32GB RAM for processing the lion dataset ...

Thanks for the reply. My point is, depth and color do be processed at the same level, but cam_ may be not. More specifically, cam_.width() != depth.cols. Thus the check inside Camera::project(const Vec3f &pt, Vec2f &pt2f, Vec2i &pt2i) is invalid.

  • In Camera::project(const Vec3f &pt, Vec2f &pt2f, Vec2i &pt2i) const in camera.cpp, it is checked that pt2i is within width and height, returning false otherwise.

@dolphin-li Thanks a lot for explaining, I think I understood what you meant and have hopefully made a (maybe hacky) fix for this.
Can you please check out the current master and verify that it doesn't crash anymore for you, if possible?

Thanks. I had made a similar fix. I think the fix you provided should work.