/extrinsic_lidar_camera_calibration

Primary LanguageMATLAB

extrinsic_lidar_camera_calibration

Overview

This is a package for extrinsic calibration between a 3D LiDAR and a camera, described in paper: Improvements to Target-Based 3D LiDAR to Camera Calibration (PDF). We evaluated our proposed methods and compared them with other approaches in a round-robin validation study, including qualitative results and quantitative results, where we use image corners as ground truth to evaluate our projection accuracy.

[Super Quick Start] To run the testing code, please clone this repo, place the test datasets in folders, change the two paths in main.m, and then hit run!

  • Authors: Bruce JK Huang and Jessy W. Grizzle
  • Maintainer: Bruce JK Huang, brucejkh[at]gmail.com
  • Affiliation: The Biped Lab, the University of Michigan

This package has been tested under MATLAB 2019a and Ubuntu 16.04. More detailed introduction will be updated in a week. Sorry for the inconvenience!

Abstract

The homogeneous transformation between a LiDAR and monocular camera is required for sensor fusion tasks, such as SLAM. While determining such a transformation is not considered glamorous in any sense of the word, it is nonetheless crucial for many modern autonomous systems. Indeed, an error of a few degrees in rotation or a few percent in translation can lead to 20 cm translation errors at a distance of 5 m when overlaying a LiDAR image on a camera image. The biggest impediments to determining the transformation accurately are the relative sparsity of LiDAR point clouds and systematic errors in their distance measurements. This paper proposes (1) the use of targets of known dimension and geometry to ameliorate target pose estimation in face of the quantization and systematic errors inherent in a LiDAR image of a target, and (2) a fitting method for the LiDAR to monocular camera transformation that fundamentally assumes the camera image data is the most accurate information in one's possession.

Quick View

Using the obtained transformation, LiDAR points are mapped onto a semantically segmented image. Each point is associated with the label of a pixel. The road is marked as white; static objects such buildings as orange; the grass as yellow-green, and dark green indicates trees.

Why important?

A calibration result is not usable if it has few degrees of rotation error and a few percent of translation error. The below shows that a calibration result with little disturbance from the well-aigned image.

Why this package? (coming strong in a week)

TODO

Overall pipeline (coming strong in a week)

TODO

Presentation and Video (coming strong in a week)

https://www.brucerobot.com/

Installation

  • Which toolboxes are used in this package:
    • MATLAB 2019a
    • optimization_toolbox
    • phased_array_system_toolbox
    • robotics_system_toolbox
    • signal_blocks
  • Dataset: download from here.

Dataset

Please download point cloud mat files from here and put them into LiDARTag_data folder.

Please downlaod bagfiles from here and put them into bagfiles folder.

Running

[quick start] To run the testing code, please clone this repo, place the test datasets in folders, change the two paths in main.m, and then hit run!

[Dataset structure] Put ALL information of datasets into getBagData.m. This funciton returns two data structure: TestData and BagData.

  • TestData contains bagfile and pc_file, where bagfile is the name of the bagfile and pc_file is mat files of FULL scan of point cloud.
  • BagData contatins:
    • bagfile: name of the bagfile
    • num_tag: how many tags in this dataset
    • lidar_target
      • pc_file: the name of the mat file of this target of point cloud
      • tag_size: size of this target
    • camera_target
      • corners: corner coordinates of the camera targets

Important Parameters (coming strong in a week)

TODO

Examples (coming strong in a week)

TODO

Qualitative results

For the method GL_1-R trained on S_1, the LiDAR point cloud has been projected into the image plane for the other data sets and marked in green. The red circles highlight various poles, door edges, desk legs, monitors, and sidewalk curbs where the quality of the alignment can be best judged. The reader may find other areas of interest. Enlarge in your browser for best viewing.

Quantitative results

For the method GL_1-R, five sets of estimated LiDAR vertices for each target have been projected into the image plane and marked in green, while the target's point cloud has been marked in red. Blowing up the image allows the numbers reported in the table to be visualized. The vertices are key.

Citations

The detail is described in:

Jiunn-Kai Huang and J. Grizzle, "Improvements to Target-Based 3D LiDAR to Camera Calibration" (PDF)(arXiv)

@article{huang2019improvements,
  title={Improvements to Target-Based 3D LiDAR to Camera Calibration},
  author={Huang, Jiunn-Kai and Grizzle, Jessy W},
  journal={arXiv preprint arXiv:1910.03126},
  year={2019}
}