Extrinsics in RGBD integration

[Willyzw]

Hallo, while trying out the example for RGB-D image based integration, I notice that the extrinsic/global poses are inverted. Since normally the global poses are given as camera to world, shall I interpret the inverted pose as world to camera transformation?

Lets take an example, the global pose is inverted here for scannet dataset.

vdbfusion/examples/python/datasets/scannet.py

Line 33 in 40ad302

pose = np.linalg.inv(pose)

Then the RGBD point cloud is transformed using the inverted pose here

vdbfusion/examples/python/datasets/scannet.py

Lines 61 to 63 in 40ad302

	pcd = o3d.geometry.PointCloud.create_from_rgbd_image(
	rgbd, intrinsic, pose, project_valid_depth_only=True
	)

So far is fine, as I vaguely remember open3d indeed assumes an extrinsic from world to camera. However, this inverted pose is further fed to the integration pipeline then apparently regarded as the frame origin. I am wondering whether this is really the purpose?

vdbfusion/examples/python/datasets/scannet.py

Line 74 in 40ad302

return xyz, np.array(pose)

[nachovizzo]

Hello @Willyzw,

Honestly, I can't recall at this exact moment why we inverted the pose there. I guess because the scannet poses might be given on the world to the camera frame. In any case, this is a totally valid question, the good news is that if this is not working for your application you only need to change the "application code", which in this example is a data loader.

The VDBFusion pipeline only expects globally aligned point clouds:

vdbfusion/src/vdbfusion/vdbfusion/VDBVolume.h

Line 17 in 40ad302

/// @brief Integrates a new (globally aligned) PointCloud into the current

How you get those point clouds, is not of the concern of the pipeline ;)

Please let me know if this answers your question.

[Willyzw]

Thanks for the reply. Yes it solve my confusion. The interface expects the point clouds as well as the origins in global frame. Thus by contrast to open3d, which assumes an extrinsic pose from global to local frame, the dataloader needs to return normal uninverted pose.

[ShrutheeshIR]

Hey @Willyzw , I'm also trying to use an RGBD sensor instead of Lidar. Could you give a rough outline of steps you used. Thanks!