/limo

Lidar-Monocular Visual Odometry

Primary LanguageC++GNU General Public License v3.0GPL-3.0

limo

Lidar-Monocular Visual Odometry. This library is designed to be an open platform for visual odometry algortihm development. We focus explicitely on the simple integration of the following key methodologies:

  • Keyframe selection
  • Landmark selection
  • Prior estimation
  • Depth integration from different sensors.
  • Scale integration by groundplane constraint.

The core library keyframe_bundle_adjustment is a backend that should faciliate to swap these modules and easily develop those algorithms.

  • It is supposed to be an add-on module to do temporal inference of the optimization graph in order to smooth the result

  • In order to do that online a windowed approach is used

  • Keyframes are instances in time which are used for the bundle adjustment, one keyframe may have several cameras (and therefore images) associated with it

  • The selection of Keyframes tries to reduce the amount of redundant information while extending the time span covered by the optimization window to reduce drift

  • Methodologies for Keyframe selection:

    • Difference in time
    • Difference in motion
  • We use this library for combining Lidar with monocular vision.

  • Limo2 on KITTI is LIDAR with monocular Visual Odometry, supported with groundplane constraint.

Details

This work was accepted on IROS 2018. See https://arxiv.org/pdf/1807.07524.pdf . Until publication in proceedings of IROS, cite arxiv article

@ARTICLE{2018arXiv180707524G, author = {{Graeter}, J. and {Wilczynski}, A. and {Lauer}, M.}, title = "{LIMO: Lidar-Monocular Visual Odometry}", journal = {ArXiv e-prints}, archivePrefix = "arXiv", eprint = {1807.07524}, primaryClass = "cs.RO", keywords = {Computer Science - Robotics, Electrical Engineering and Systems Science - Image and Video Processing}, year = 2018, month = jul, adsurl = {http://adsabs.harvard.edu/abs/2018arXiv180707524G}, adsnote = {Provided by the SAO/NASA Astrophysics Data System} }

Please note that LIMO2 differs from the publication. We enhanced the speed a little and added additional groundplane reconstruction for pure monocular visual odometry and a combination of scale from LIDAR and the groundplane (best performing on KITTI).

Installation

Requirements

In any case:

 sudo apt-get install libpng++-dev
sudo apt-get install python-catkin-tools
  • install opencv_apps:
sudo apt-get install ros-kinetic-opencv-apps
  • install git:
sudo apt-get install git

Build

  • initiate a catkin workspace:

    cd ${your_catkin_workspace}
    catkin init
  • clone limo into src of workspace:

    mkdir ${your_catkin_workspace}/src
    cd ${your_catkin_workspace}/src
    git clone https://github.com/johannes-graeter/limo.git
  • clone dependencies and build repos

    cd ${your_catkin_workspace}/src/limo
    bash install_repos.sh
  • unittests:

    cd ${your_catkin_workspace}/src/limo
    catkin run_tests --this --profile limo_release

Run

  • get test data from https://www.mrt.kit.edu/graeterweb/04.bag

    1. this is a bag file generated from Kitti sequence 04 with added semantic labels.
    2. there is more under the same address all named ??.bag (supported: 00.bag, 01.bag, 04.bag)
  • in different terminals

    1. roscore
    2. rosbag play 04.bag -r 0.1 --pause --clock
    3. source ${your_catkin_workspace}/devel_limo_release/setup.sh
      roslaunch demo_keyframe_bundle_adjustment_meta kitti_standalone.launch
    4. unpause rosbag (hit space in terminal)
    5. rviz ${your_catkin_workspace}/src/demo_keyframe_bundle_adjustment_meta/res/default.rviz
  • watch limo trace the trajectory in rviz :)

  • Before submitting an issue, please have a look at the section Known issues.

Todo

  • runtime is ok for individual modules, however communication between nodes must be enhanced to ensure online usage (nodelets...).
  • Make it overall faster.
  • Test on our own test car.

Try it out

If you just want to give it a quick peek, I prepared a ready-to-use virtualbox image (packed with Ubuntu 16.04.04, ros kinetic and all dependencies for limo).

  • download it from https://www.mrt.kit.edu/graeterweb/limo_full.ova.
  • Password for the vm-image is "1234".
  • Find all modules in ~/workspaces/limo/ .
  • Run example (~/04.bag) as described above.
  • Note that the runtime in the virtual machine is slower than on a normal system.

Known issues

  • Unittest of LandmarkSelector.voxel fails with libpcl version 1.7.2 or smaller (just 4 landmarks are selected). Since this works with pcl 1.8.1 which is standard for ros melodic, this is ignored. This should lower the performance of the software only by a very small amount.