Question regarding the unit of the depth returned by "unbatched_raytrace"

[hubert0527]

Hi,

I am facing some issues understanding the unit of the depth returned by unbatched_raytrace. I originally thought the value should use the voxel size as its unit (e.g., a depth=2 implies the hit point is 2 voxels away from the camera origin). But I recently found some weird cases that the returned depth is much shorter than I expected. Note that I am on the latest commit 76b38db , so the rendering issue mentioned in #490 should have been fixed.

What I see
For instance, I rendered the depth of a scene (note that the voxel size is 25cm) as shown below. The maximum depth (excluding sky) is 6.4022145, which does not make sense at all, as the building should be at least 20 meters away from the camera.

The depth should correspond to the following image:

What I did
I render the depth with the following code:

ridx, pidx, depth = spc_render.unbatched_raytrace(
    self.octree, self.points, self.pyramid, self.prefix,
    rays.origins, rays.dirs, level, return_depth=True, with_exit=with_exit)

Printing the values I get:

(Pdb) self.points[pidx.long()].shape
torch.Size([31070, 3])
(Pdb) rays.origins[ridx.long()].shape
torch.Size([31070, 3])
(Pdb) depth.shape
torch.Size([31070, 2])

(Pdb) rays.origins[ridx.long()]
tensor([[ 4.4544, -0.2413,  3.5482],
        [ 4.4544, -0.2413,  3.5482],
        [ 4.4544, -0.2413,  3.5482],
        ...,
        [ 4.4544, -0.2413,  3.5482],
        [ 4.4544, -0.2413,  3.5482],
        [ 4.4544, -0.2413,  3.5482]], device='cuda:0')
(Pdb) self.points[pidx.long()]
tensor([[254, 250, 129],
        [253, 250, 128],
        [253, 251, 128],
        ...,
        [253,   0, 250],
        [253,   0, 249],
        [252,   0, 249]], device='cuda:0', dtype=torch.int16)
(Pdb) depth
tensor([[5.0902, 5.1003],
        [5.1017, 5.1017],
        [5.1017, 5.1115],
        ...,
        [4.3957, 4.4042],
        [4.4042, 4.4056],
        [4.4056, 4.4062]], device='cuda:0')

[In case this info is needed]
(Pdb) rays.dirs[ridx.long()]
tensor([[-0.6802,  0.2356, -0.6942],
        [-0.6802,  0.2356, -0.6942],
        [-0.6802,  0.2356, -0.6942],
        ...,
        [-0.7894, -0.1722, -0.5892],
        [-0.7894, -0.1722, -0.5892],
        [-0.7894, -0.1722, -0.5892]], device='cuda:0')

This implies this raytrace step obtains 31070 intersection points. For the very first intersection, the ray origin is around [4, 0, 3], intersect with the voxel at [254, 250, 129], and the returned depth is 5.0902. But I do not understand the underlying meaning of this 5.0902 value. Note that this is a depth 8 octree, and I only render at level=8.

Thanks!

[Caenorst]

Hi @hubert0527,

great question by SPC raytracing documentation the SPC is actually in the [-1, 1] range.

With this in mind that mean the voxel [254, 250, 129] is actually at position [0.9921568627450981, 0.9607843137254901, 0.0117647058823529].

The explanation of why we choosed to normalize the SPC is that depth should modify the resolution not the range of the data (going from level 8 to 9 should just modify the size of a voxel, not make the voxelgrid from [0, 255] to [0, 1024]), with this the depth returned should be in the same range regardless of the level you choose

[hubert0527]

Thank you for the clarification! That makes a lot more sense to me!