Issue regarding correctness of rendered depth images.

[touranisatyajit]

Hi,

Thank you for the wonderful rendering code. :)

I have used your code to render shapenet chair models by following the instructions given in Readme.

I also tried to check for the correctness of the depth image using 2d -> 3d (Back projection) using the formula

x3d = depth_value * K_inverse * x2d

where x2d is the 2d point, x3d is the 3d point and K_inverse is the inverse of the camera intrinsic matrix(K matrix) obtained from this link.click here

However I noticed that the 3d point cloud was not accurate.

Below are the images of the point cloud from different viewpoints as well as the rendered depth image of a particular chair using your script.

The link above which gives camera intrinsics has two different code snippets which give two different camera matrices(differ only in the focal_y value).
I tried reprojecting using both camera matrices. I also tried modifying the depth scale in your script with different values, and yet the point cloud generated was not accurate.

Could you help me with this? Could there be a case where I need to perform some additional operations on the depth values.?
There could also be the case where the camera intrinsics are wrong, but I have tried generating camera intrinsics(K matrix) from 3-4 sources and all of them were giving the similar camera intrinsics and none of them helped to get an accurate point cloud. I don't have a method to prove whether or not the camera intrinsics are wrong.

Also the camera extrinsics should not affect the rendered image or point cloud, they only affect the orientation and position, but still I tried adding them to the reprojection calculation but it did not make a difference.

Hope you are able to help me regarding this by telling me what changes could be useful in my case(either in the rendering scipt to get the depth images or otherwise).

[panmari]

Could this be related to quanitization? I think the default output quantizes depth to values in [0, 255].

Did you also try this with --format=exr, where values are not quantized?

[touranisatyajit]

Hi,

I tried changing the format to EXR but the pointcloud still looks pretty much the same.
Could there be an issue with the values in the .obj files probably?

[touranisatyajit]

Resolving this issue. I dont think the fault lies in the script.

[yinyunie]

Resolving this issue. I dont think the fault lies in the script.

Use OPEN_EXR to output the depth result. BTW, the values in the depth maps from blender store the distance (not the z coordinates) from the camera to point clouds.