facebookresearch/PyTouch

Visualizing depth info

Closed this issue · 0 comments

I am trying to visualize the depth output by running examples/surface_3d.py. The code I run:

  sequence = img_seq_ds[0]
  for img in sequence:
      output = surface3d.point_cloud_3d(img_color=img)
      visualizer.render(output.depth)
      print(output.depth)

It nicely visualizes the point_cloud_3d, but it is not working for depth( according to the comments provided by authors, it must work). The error I get:

Error executing job with overrides: []
Traceback (most recent call last):
  File "/home/shuk/digits2/pytouch_digit/examples/surface_3d.py", line 78, in visualize_surface_3d
    visualizer.render(output.depth)
  File "/home/shuk/anaconda3/envs/digits2/lib/python3.8/site-packages/pytouch-0.4.2-py3.8.egg/pytouch/common/visualizer.py", line 259, in render
    clouds = self.points_to_cloud(points_3d=[points_3d])
  File "/home/shuk/anaconda3/envs/digits2/lib/python3.8/site-packages/pytouch-0.4.2-py3.8.egg/pytouch/common/visualizer.py", line 247, in points_to_cloud
    o3d.utility.Vector3dVector(points_3d_np.transpose())
RuntimeError: Unable to cast Python instance to C++ type (compile in debug mode for details)

Moreover, if i just print the depth values without any visualization, I get arrays filled with zeros:

 [0. 0. 0. ... 0. 0. 0.]
 [0. 0. 0. ... 0. 0. 0.]
 [0. 0. 0. ... 0. 0. 0.]]
[[0. 0. 0. ... 0. 0. 0.]
 [0. 0. 0. ... 0. 0. 0.]
 [0. 0. 0. ... 0. 0. 0.]
 ...
 [0. 0. 0. ... 0. 0. 0.]
 [0. 0. 0. ... 0. 0. 0.]
 [0. 0. 0. ... 0. 0. 0.]]

My final goal is to actually get the depth values and apply PCA to get principal axis of the object for ellipse estimation, as desribed on this paper from MIT's GelSight: Cable Manipulation with a Tactile-Reactive Gripper (check Cable Pose Estimation).
It would be even better if we could have a 20-30 min Skype call with the authors to discuss further questions.Thanks!