vzyrianov/lidargen

Image to point cloud

Opened this issue · 3 comments

zisismp4 commented

Hello and thanks for sharing this great work! I am quite new with the LiDAR technology so please excuse my possibly naive question. After sampling, the output is 2-channel images. How can I recreate the pointcloudfrom them?

vzyrianov commented

Hello Zisis, thanks for checking out our work!

Each pixel value in a range image encodes a distance value. The {x,y} coordinate of that pixel contains information about the corresponding LiDAR point's {yaw, pitch} value. Given N x {depth, yaw, pitch} it is possible to project the points into 3D point cloud Nx3.

You can use this function in the codebase to do this transformation (transposing the input may be required) https://github.com/vzyrianov/lidargen/blob/4c3226cd7a53c241320923f936a434fdda3bad59/metrics/histogram/histogram.py#LL20C3-L20C3

  • 👍1
zisismp4 commented

Thanks @vzyrianov for the quick response! I am a little bit confused! Let me describe what I am doing:

  1. I downloaded the pretrained model and the 3D LiDAR readings (just on folder because of memory issue)
  2. I ran python lidargen.py --sample --exp kitti_pretrained --config kitti.yml and two things were generated: image_grid.png and samples.pth in kitti_pretrained/image_samples/images/ and 0-7.pth in kitti_pretrained/unconditional_samples

Questions:

  • What is the difference between samples.pth and 0-7.pth ?
  • When I tried the range_image_to_point_cloud_fast with 0.pth, I got an error:

Exception has occurred: ValueError
operands could not be broadcast together with shapes (65536,) (131072,)
File "/home/zisis/Documents/lidargen/metrics/histogram/histogram.py", line 44, in range_image_to_point_cloud_fast
pts[:, 0] = np.cos(yaw) * np.cos(pitch) * depth
File "/home/zisis/Documents/lidargen/samples_load.py", line 34, in
pcd = ri_2_pcd(un_samples_pth.numpy())
ValueError: operands could not be broadcast together with shapes (65536,) (131072,)

vzyrianov commented
  • The numbered .pth files are the generated samples (there will be multiple files, each one containing a batch Bx2x64x1024). samples.pth will contain the last generated batch.
  • Pass in only the first color channel (the second color channel is lidar intensity).