/HeLiMOS-PointCloud-Toolbox

HeLiMOS: Heterogeneous LiDAR Dataset for Moving Object Segmentation @ IROS'2024

Primary LanguageC++

🚗 HeLiMOS Pointcloud Toolbox 🏃



Video   •   Install   •   Paper   •   Contact Us

animated

The HeLiMOS pointcloud toolbox is a data processing software for moving object segmentation (MOS) in the HeLiPR dataset.
It includes an effective merging-and-splitting-based approach ((a) and (g) in the upper figure) for labeling four heterogeneous LiDAR sensors.

📦 Installation

What we need are just minimal dependencies.

sudo apt-get install g++ build-essential libpcl-dev libeigen3-dev python3-pip python3-dev cmake -y

Next, clone and compile the HeLiPR-Pointcloud-Toolbox repository using git as follows:

git clone https://github.com/url-kaist/HeLiMOS-PointCloud-Toolbox.git
cd HeLiMOS-Pointcloud-Toolbox
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release && make -j 16

🔍 Overview

This toolbox consists of three modules: helimos_saver, helimos_merger, and helimos_propagator!

  • helimos_saver deskews and saves individual LiDAR data and pose data in the HeLiMOS format.

  • helimos_merger synchronizes and combines the saved LiDAR data into a single merged cloud.
Aeva
Aeva cloud
Livox
Livox cloud
Merged
Merged cloud
Velodyne
Velodyne cloud
Ouster
Ouster cloud
  • helimos_propagator backpropagates the labeled points from the merged cloud to the individual clouds.
Propagated
Labeled merged cloud
Aevadyn
Labeled Aeva cloud
Livoxdyn
Labeled Livox cloud
Velodyn
Labeled Velodyne cloud
OSdyn
Labeled Ouster cloud

🔧 Usage

1. Getting Started with the HeLiPR Dataset

When you download a specific sequence of the HeLiPR dataset, you can see the following file structure:

${sequence.tar.gz} # /path/to/HeLiPR/sequence
├── LiDAR
│   ├── Aeva
│   │   ├── <timestamp>.bin
│   │   ├── ...
│   ├── Avia
│   │   ├── <timestamp>.bin
│   │   ├── ...
│   ├── Ouster
│   │   ├── <timestamp>.bin
│   │   ├── ...
│   └── Velodyne
│       ├── <timestamp>.bin
│       ├── ...
├── LiDAR_GT
│   ├── Aeva_gt.txt
│   ├── Avia_gt.txt
│   ├── Ouster_gt.txt
│   └── Velodyne_gt.txt
├── Calibration
├── Inertial_data
└── stamp.csv
  • LiDAR : the folder contains LiDAR point cloud, with filenames structured as <timestamp>.bin.
  • LiDAR_GT : the folder contains individual LiDAR ground truth poses based on the INS system, in a TXT format structured as [timestamp, x, y, z, qx, qy, qz, qw].
    For more detailed information, please refer to the HeLiPR website.

2. Before Running HeLiMOS toolbox

Refer to the config-helimos.yaml file in config folder, where you can find the following parameters:

Path:
  binPath: "/path/to/HeLiPR/sequence/LiDAR/"
  trajPath: "/path/to/HeLiPR/sequence/LiDAR_GT/" 
  savePath: "/path/to/HeLiMOS/sequence/Deskewed_LiDAR"
  • binPath: LiDAR folder path in the HeLiPR dataset
  • trajPath: LiDAR_GT folder path in the HeLiPR dataset
  • savePath: path where the HeLiMOS dataset will be saved.

binPath and trajPath should be set to the actual paths of the HeLiPR dataset, and the savePath should be set to the folder where you want to save the HeLiMOS dataset.

Note! Except for changing the path settings to your path, please do not change any other settings.

3. HeLiMOS saver: Converting HeLiPR data format to HeLiMOS data format

helimos_saver converts the HeLiPR data format to the HeLiMOS data format. HeLiMOS data format is similar to the SemanticKITTI format. For more detailed information, please refer to the SemanticKITTI website.

if you want to use helimos_saver,

./helimos_saver

When you run helimos_saver, the deskewed point clouds will be saved in the velodyne folder. and individual LiDAR poses will be saved as a poses.txt file which contains the first 3 rows of a 4x4 homogeneous pose matrix [r11 r12 r13 tx r21 r22 r23 ty r31 r32 r33 tz].

And all calibration values are identity matrices, saved as calib.txt.

${savePath} # /path/to/HeLiMOS/sequence/Deskewed_LiDAR
├── Aeva
│   ├── calib.txt
│   ├── poses.txt
│   └── velodyne
│     ├── 000000.bin
│     ├── 000001.bin
│     ├── ...
├── Avia
│   ├── calib.txt
│   ├── poses.txt
│   └── velodyne
│     ├── 000000.bin
│     ├── 000001.bin
│     ├── ...
├── Ouster
│   ├── calib.txt
│   ├── poses.txt
│   └── velodyne
│     ├── 000000.bin
│     ├── 000001.bin
│     ├── ...
└── Velodyne
    ├── calib.txt
    ├── poses.txt
    └── velodyne
      ├── 000000.bin
      ├── 000001.bin
      ├── ...

4. HeLiMOS merger: Synchronizing and combining individual clouds into merged cloud

helimos_merger synchronizes and merges four individual point clouds into a single, unified point cloud. When you run this tool, the four point clouds captured around the same timestamp are merged into the Ouster frame and saved in the HeLiMOS format within a folder named Merged.

if you want to use helimos_merger,

./helimos_merger

Then you will obtain the following Merged scan data.

${savePath} # e.g., /path/to/HeLiMOS/sequence
├── Aeva
├── Avia
├── Ouster
├── Velodyne
└── Merged
    ├── calib.txt
    ├── poses.txt
    └── velodyne
      ├── 000000.bin 
      ├── 000001.bin
      ├── ...

5. HeLiMOS propagator : Backpropagating the labeled merged cloud onto individual clouds

helimos_propagator back-propagates MOS labels from the Merged scan to the four individual LiDAR scans. The MOS labels from the Merged scan should be saved as a uint32_t type .label file like SemanticKITTI, and the label classes must follow the SemanticKITTI MOS classes.

This means that each label file should only include the following classes:

  • 0: unlabeled
  • 9: static
  • 251: moving

Save the MOS .label files in the Merged - labels folder, as shown below:

${savePath} # e.g., /path/to/HeLiMOS/sequence
├── Aeva
├── Avia
├── Ouster
├── Velodyne
└── Merged
    ├── calib.txt
    ├── poses.txt
    ├── velodyne
    │  ├── 000000.bin 
    │  ├── 000001.bin
    │  ├── ...
    └── labels
       ├── 000000.label
       ├── 000001.label
       ├── ...

Then, run the helimos_propagator.

./helimos_propagator

Then you can get the propagated MOS labels for the four LiDARs.

${savePath} # e.g., /path/to/HeLiMOS/sequence
├── Aeva
│    ├── calib.txt
│    ├── poses.txt
│    ├── velodyne
│    │  ├── 000000.bin 
│    │  ├── 000001.bin
│    │  ├── ...
│    └── labels
│       ├── 000000.label
│       ├── 000001.label
│       ├── ...
├── Avia
│    ├── calib.txt
│    ├── poses.txt
│    ├── velodyne
│    │  ├── 000000.bin 
│    │  ├── 000001.bin
│    │  ├── ...
│    └── labels
│       ├── 000000.label
│       ├── 000001.label
│       ├── ...
├── Ouster
│    ├── calib.txt
│    ├── poses.txt
│    ├── velodyne
│    │  ├── 000000.bin 
│    │  ├── 000001.bin
│    │  ├── ...
│    └── labels
│       ├── 000000.label
│       ├── 000001.label
│       ├── ...
├── Velodyne
│    ├── calib.txt
│    ├── poses.txt
│    ├── velodyne
│    │  ├── 000000.bin 
│    │  ├── 000001.bin
│    │  ├── ...
│    └── labels
│       ├── 000000.label
│       ├── 000001.label
│       ├── ...
└── Merged
    ├── calib.txt
    ├── poses.txt
    ├── velodyne
    │  ├── 000000.bin 
    │  ├── 000001.bin
    │  ├── ...
    └── labels
       ├── 000000.label
       ├── 000001.label
       ├── ...

License and Citation

@misc{lim2024helimos,
      title={HeLiMOS: A Dataset for Moving Object Segmentation in 3D Point Clouds From Heterogeneous LiDAR Sensors}, 
      author={Hyungtae Lim and Seoyeon Jang and Benedikt Mersch and Jens Behley and Hyun Myung and Cyrill Stachniss},
      year={2024},
      eprint={2408.06328},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
}

@misc{jung2023helipr,
      title={HeLiPR: Heterogeneous LiDAR Dataset for inter-LiDAR Place Recognition under Spatial and Temporal Variations}, 
      author={Minwoo Jung and Wooseong Yang and Dongjae Lee and Hyeonjae Gil and Giseop Kim and Ayoung Kim},
      year={2023},
      eprint={2309.14590},
      archivePrefix={arXiv},
      primaryClass={cs.RO}
}

Copyright Notice

All point clouds are copyrighted by SNU RPM Labs and MOS labels are copyrighted by KAIST Urban Robotics Lab, and those are distributed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 License. This license requires proper attribution to the author for any use, prohibits commercial usage, and mandates that derivative works be licensed similarly.

Maintainer

  • Seoyeon Jang (9uantum01 at kaist dot ac dot kr)
  • Hyungtae Lim (shapelim at mit dot edu)