/PointcloudMeshing

Primary LanguagePython

PointCloud Meshing

This package has the objective to obtain a 3D reconstruction of a car, starting from the outputs of one or more depth cameras, by applying filters and ICP algorithms.

Nodes:

  • realsense_streamer.py:

This node acts as a ROS wrapper for RealSense cameras by using the pyrealsense2 library. The following topics are published during the runtime of the node:

  • /camera/color
  • /camera/depth
  • /camera/depth/color
  • /camera/infrared
  • /camera/points
  • /camera/camera_info
  • /camera/IMU

All the topics are published directly, exception made for the /camera/points topic, which, being the pointcloud, is preprocessed and then published. To launch the node, a launchfile has been made, roslaunch pc_meshing realsense.launch, which opens this node and rviz with its relative configuration to correctly visualize the pointcloud. Two launch parameters have been implemented at the moment:

  • stream: if True, the streaming starts from the camera, if connected
  • bag_filename: when stream is False, the streaming is enabled from a bag file; this is the filepath

Lastly, a simple GUI is provided when launching the node: it has the purpose of letting the user enable/disable filters and modify their values. It also is able to pause or resume the stream. The filters implemented at the moment are:

  • Decimation Filter
  • Threshold Filter
  • Spatial Filter
  • Temporal Filter
  • Hole Filling Filter
  • Depth To Disparity

  • kinect_streamer.py:

It acts as a ROS wrapper for the Kinect v2, relying on freenect2 and libfreenect2 libraries. The following topics are published during runtime:

  • /kinect/color
  • /kinect/depth
  • /kinect/ir
  • /kinect/points

  • noise_analyzer.py:

A simple node created to help visualize PointClouds noise. It subscribes on the streamer /points/ topic (either realsense or kinect) and collects points in a user defined section. The section is then sent back to /square/points, to check everything is fine, and a 3D plot is generated.

  • registration.py:

This node has the purpose to take the pointclouds published by either realsense_streamer.py or kinect_streamer.py, save them in a buffer, perform reconstruction ad publish the final result. When starting the node with rosrun pc_meshing registration.py, the streamer node will slow its publishing rate to 1 frame per second, thus skipping frames. This is to mimic sampling on the stream. If 'Start Registration' is clicked, the frames will be saved in a buffer defined by the user via the first spinbox. During buffer filling, every element in the buffer is temporarly sent on /computed/points. This helps seeing what actually is in the buffer. When the buffer is full, the user can click on 'Compute Registration'; this will apply the following ICP pipeline on the pointclouds:

  • Coarse ICP Point to Plane
  • Fine ICP Point to Plane
  • Coarse ICP Point to Point
  • Coarse Colored ICP The second spinbox is used to choose a voxel size: this is how much the pointclouds in the buffer will be downsized and will affect precision and speed. After the pipeline, an open3d PoseGraph is created; if the 'Fine Registration' checkbutton is active, open3d.pipelines.registration.global_optimization() is performed to refine it. Then the transformations in the PoseGraph are applied to the pointclouds in the buffer, which are then merged and published in /computed/points. If 'Random Color' is active, every pointcloud in the buffer will have different random colors. This is useful when trying to see the different pointclouds.

Changelog:

  • Update 2 - 10/12/2022 - Version 1.2.0:

  • realsense_streamer.py:

  • Changed node name from streamer.py to realsense_streamer.py
  • Added Depth To Disparity filter (only works with D series cameras)
  • Filters application is now handled by the filters_pipeline() function

  • kinect_streamer.py:

  • Added node
  • Added color, depth and ir streaming
  • Added pointcloud streaming on /kinect/points

  • registration.py:

  • Added outlier filtering option before PoseGraph computation
  • Added non finite points filtering (Kinect gives error without filtering)
  • Now during buffer filling, fragments are sent to /computed/points
  • Save function added
  • When started now the node automatically subscribes to either /kinect/points or /camera/points

  • noise_analyzer.py:

  • Added node
  • Node automatically check which topic to subscribe (either realsense or kinect)
  • The isolated section is sent to /square/points

  • realsense_libs:

  • Changed libraries name from streamer_libs to realsense_libs
  • Added filters pipeline function

  • kinect_libs:

  • Added library
  • Added custom Queue for frame listener
  • Added function that handles frames
  • Added function that casts from points array and image to open3d PointCloud

  • icp:

  • Added outliers filtering function

  • Update 1 - 03/10/2022:

  • streamer.py:

  • Added Temporal Filter
  • Added Hole Filling Filter
  • Added IMU data publishing
  • Streamer fps now change wether or not the registration.py node is up or not

  • registration.py:

  • Added node
  • Custom PointCoud class created
  • Fast multiway registration via open3d posegraphs

  • streamer_libs:

  • pyrealsese2 frame to ROS Imu message bridge function added

  • GUI:

  • Added check() method to Checkbutton class
  • Added Button class
  • Added Spinbox class

  • shared_libs:

  • Added library
  • Added function that check wether or not a node is active
  • Added custom ROS PointCloud2 message to open3d pointcloud bridge function