/APMP

[ICRA'24] Accurate Prior-centric Monocular Positioning with Offline LiDAR Fusion

Primary LanguageC++GNU General Public License v3.0GPL-3.0

APMP

This is a repository of the online localization module of:

[ICRA 24] Accurate Prior-centric Monocular Positioning with Offline LiDAR Fusion

  • A hierarchical visual tracking of SuperPoint features.
  • Visual map is compatible with COLMAP format.

Contributers: Jinhao He *, Huaiyang Huang *, Shuyang Zhang, Jianhao Jiao, Chengju Liu and Ming Liu.

{*} for equal contribution

ICRA24-Presentation

example result (may need some time to load the gif):

icra24-gif1-crop

Build the project

prerequisites

ubuntu 20.04, ros-noetic, cuda-11.7, cudnn-8.5 libgoogle-glog-dev faiss

clone repo

under a catkin workspace

mkdir src
cd src
git clone https://github.com/David-Willo/APMP.git --recursive

due to lfs size limit, download extra dependencies to the dependency folder

  • dependencies:
    • torch_catkin: catkin wrapper for libtorch and torch_tensorrt repo
    • xslam_test_data: model files and sample data download and extract

build

under catkin workspace

catkin init
catkin config --extend /opt/ros/noetic
catkin config --merge-devel
catkin config --cmake-args -DCMAKE_BUILD_TYPE=Release
catkin build sophus
catkin build xslam_visual_localization

prepare data and run the script

  • Download demo data and extract to datapath,

    • the map database is compatible with the colmap sparse reconstruction format,
    • feature extraction and database generation can follow hloc demo

  • update parameters in runner script (xslam/script/run_apmp.sh) and execute,

known issues:

build error due to protobuf version mismatch.

update the CMakeList.txt of protobuf_catkin

set(USE_SYSTEM_PROTOBUF "OFF") to compile with given version

undefined reference to `google::kLogSiteUninitialized'

update dependencies/glog_catkin/cmake/Modules/FindGlog.cmake

set(GLOG_PREFER_EXPORTED_GLOG_CMAKE_CONFIGURATION False)

or

build & install an old version of glog (e.g. v0.5.0-rc2)

according to issue543

hardware/model dimension mismatch

use /script/convert_model to compile your own model file

Acknowledgements

Our implementation refers to ORB-SLAM2, hloc, SuperPointPretrainedNetwork, maplab, DLIO We really appreciate these open source projects!

Timeline

  • Early Release (2024-07-10)
  • Beta Release (TODO)

Related Papers

If our work has helped you, please cite:

@misc{he2024accuratepriorcentricmonocularpositioning,
      title={Accurate Prior-centric Monocular Positioning with Offline LiDAR Fusion}, 
      author={Jinhao He and Huaiyang Huang and Shuyang Zhang and Jianhao Jiao and Chengju Liu and Ming Liu},
      year={2024},
      eprint={2407.09091},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2407.09091}, 
}