VECtor Benchmark is the first complete set of benchmark datasets captured with a multi-sensor setup containing an event-based stereo camera, a regular stereo camera, multiple depth sensors, and an inertial measurement unit. The setup is fully hardware-synchronized and underwent accurate extrinsic calibration. All sequences come with ground truth data captured by highly accurate external reference devices such as a motion capture system. Individual sequences include both small and large-scale environments, and cover the specific challenges targeted by dynamic vision sensors.
This toolbox is a ROS workspace integrating with a set of easy-to-use dataset functions, including:
- Event Visualizer: convert raw event stream into accumulated event frames.
- Data Validator: validate the downloaded rosbag or the merged rosbag for its completeness.
- Bag Merger: merge multiple, single-topic rosbags chronologically into one complete rosbag.
- Bag Splitter: split one multi-topic rosbag into a few compressed, single-topic rosbags.
- Timeline Reconstructor: reconstruct raw recording's timeline for each data channel.
- Bag-to-HDF5 Converter: convert event stream(s) into compressed HDF5 file(s) under DSEC format.
- Bag-to-Rawfile Converter: convert all other data stream(s) into raw file(s).
- CSV-to-TUM Converter: convert OptiTrack readings into text file under TUM format.
This toolbox, together with the MPL Calibration Toolbox and the k4a Projector, is available as open-source under the terms of the BSD-3-Clause-Clear License. If you use this toolbox in an academic context, please cite the publication as follows:
@Article{gao2022vector,
author = {Gao, Ling and Liang, Yuxuan and Yang, Jiaqi and Wu, Shaoxun and Wang, Chenyu and Chen, Jiaben and Kneip, Laurent},
title = {{VECtor}: A Versatile Event-Centric Benchmark for Multi-Sensor SLAM},
journal = {IEEE Robotics and Automation Letters},
pages = {8217--8224},
volume = {7},
number = {3},
year = {2022},
doi = {10.1109/LRA.2022.3186770}
}The following instructions are tested on (A) Ubuntu 18.04 with ROS Melodic, and (B) Ubuntu 20.04 with ROS Noetic. A ROS desktop-full installation is required. On top of that, the following libraries (Eigen 3, OpenCV, Point Cloud Library) have to be installed:
sudo apt-get update
sudo apt-get install libeigen3-dev libopencv-dev libpcl-dev
ln -s /usr/include/eigen3/Eigen /usr/include/Eigen
After that, enter your catkin workspace and the build can be triggered with the following command:
cd ~/catkin_ws/src
git clone https://github.com/mgaoling/mpl_dataset_toolbox.git
cd ..
catkin_make
source ~/catkin_ws/devel/setup.bash
The event visualizer converts raw event stream into accumulated event frames. It can handle at most two different event streams.
- For monocular setup, check and modify the parameters in the
config/event_visualizer.yaml, then launch the visualization by:
roslaunch mpl_dataset_toolbox event_visualization.launch
- For stereo setup, check and modify the parameters in the
config/event_stereo_visualizer.yaml, then launch the visualization by:
roslaunch mpl_dataset_toolbox event_stereo_visualization.launch
- To visualize the rosbag, run
rqtorrqt_image_viewin another terminal to display the accumulated event frames.
The data validator helps to check the correctness and completeness of a downloaded, single-topic rosbag or a merged rosbag. It also calculates the Mean Event Rate if the event stream exists.
- Launch the validation in one terminal by:
roslaunch mpl_dataset_toolbox data_validation.launch
- Play the rosbag in another terminal by:
rosbag play [data.bag]
- Terminate the validation once the playback is over.
Note: Dropped frames on Kinect stream are to be expected, owing to the singularity of the depth calculation during recording. We have deliberately left a blank on the depth stream during the timeline reconstruction, to maintain the overall correctness.
Bag merger helps to merge different single-topic rosbags chronologically. Note that data sequences with different prefixes can be safely stored under the same directory. DO NOT MODIFY THE FILENAME!
- Put all single-topic rosbags under one folder, then launch the bag merger by:
roslaunch mpl_dataset_toolbox bag_merger.launch directory_path:=[path_to_directory]
Note: Here, only an absolute path without ~, or a relative path that is relative to this toolbox is allowed for the directory path.
Bag Splitter helps to split a multi-topic rosbag into a few single-topic rosbags, with a compression option.
- Launch the bag splitter by:
roslaunch mpl_dataset_toolbox bag_splitter.launch bag_path:=[data.bag] need_compression:=[true_or_false]
Note: Here, only an absolute path without ~, or a relative path that is relative to this toolbox is allowed for the rosbag path.
roslaunch mpl_dataset_toolbox timeline_reconstruction.launch bag_path:=[data.bag] gt_path:=[ground_truth.csv]
roscd mpl_dataset_toolbox/script
python3 bag2hdf5.py [data.bag]
roslaunch mpl_dataset_toolbox bag2rawfile.launch bag_path:=[data.bag]
roscd mpl_dataset_toolbox/script
python3 csv2tum.py [ground_truth.csv] [data.bag]