SegmentationMapping

Label pointcloud from stereo cameras or lidar with Tensorflow trained neural nets graph.pb
Build 3D Semantic voxel maps, 2D occupancy maps, 2D obstacle distance maps

The Youtube video of the Cassie Bipedal robot's perception and mapping system is powered by this library.

Here is a semantic map built on NCLT dataset

Pipeline

Two types of accepted inputs (from ROS topics):

Raw RGBD images from depth cameras, or
Lidar scans + RGB images.

C++: pcl, eigen, xmlrpcpp

Outputs (in ROS messages):

Labeled images, each pixel with a distribution across classes
Labeled PointCloud ROS message
Semantic Octomap
2D occupancy maps
2D cost map, cost computed from distance to closest obstacles

Performance

Tested on

AlienWare Laptop (running in realtime for the Cassie Blue experiement): 3FPS on i7-6820, 32GB RAM, GTX1070
Dell XPS: 5FPS on i7-7700, 32GB RAM, GTX1070.
Jetson Xavier: Around 4Hz

Dependencies

ROS: Tested on Kinetic + Ubuntu 16.04, Melodic + Ubuntu 18.04

Neural Network:

tensorflow-gpu-1.8.0
tensorrt4
cuda-9.0,
cudnn-7

Python thirdparty:

python-pcl
python-opencv

C++: pcl, eigen, OpenCV

ros thirdparty:

semantic octomap: Modified octomap, supporting Bayesian updates for semantic label fusion
octomap_ros: Put it in the same catkin workspace as the SegmentationMappping repo.
octomap_msgs: Put it in the same catkin workspace as the SegmentationMappping repo.
ros_numpy: Put it in the same catkin workspace as the SegmentationMappping repo.

Compiling

Make sure that your semantic octomap is build. Make sure that you haven't install the native octomap and octomap_ros from apt-get. Then run catkin_make install -DCMAKE_BUILD_TYPE=Release -Doctomap_DIR=${you_octomap_source_directory}/lib/cmake/octomap/ --pkg SegmentationMapping

launchfiles