APMP

This is a repository of the online localization module of:

[ICRA 24] Accurate Prior-centric Monocular Positioning with Offline LiDAR Fusion

A hierarchical visual tracking of SuperPoint features.
Visual map is compatible with COLMAP format.

Contributers: Jinhao He *, Huaiyang Huang *, Shuyang Zhang, Jianhao Jiao, Chengju Liu and Ming Liu.

{*} for equal contribution

example result (may need some time to load the gif):

Build the project

prerequisites

ubuntu 20.04, ros-noetic, cuda-11.7, cudnn-8.5 libgoogle-glog-dev faiss

clone repo

under a catkin workspace

mkdir src
cd src
git clone https://github.com/David-Willo/APMP.git --recursive

due to lfs size limit, download extra dependencies to the dependency folder

dependencies:
- torch_catkin: catkin wrapper for libtorch and torch_tensorrt repo
- xslam_test_data: model files and sample data download and extract

build

under catkin workspace

catkin init
catkin config --extend /opt/ros/noetic
catkin config --merge-devel
catkin config --cmake-args -DCMAKE_BUILD_TYPE=Release
catkin build sophus
catkin build xslam_visual_localization

prepare data and run the script

Download demo data and extract to datapath,
- the map database is compatible with the colmap sparse reconstruction format,
- feature extraction and database generation can follow hloc demo
update parameters in runner script (xslam/script/run_apmp.sh) and execute,

known issues:

build error due to protobuf version mismatch.

update the CMakeList.txt of protobuf_catkin

set(USE_SYSTEM_PROTOBUF "OFF") to compile with given version

undefined reference to `google::kLogSiteUninitialized'

update dependencies/glog_catkin/cmake/Modules/FindGlog.cmake

set(GLOG_PREFER_EXPORTED_GLOG_CMAKE_CONFIGURATION False)

build & install an old version of glog (e.g. v0.5.0-rc2)

according to issue543

hardware/model dimension mismatch

use /script/convert_model to compile your own model file

Acknowledgements

Our implementation refers to ORB-SLAM2, hloc, SuperPointPretrainedNetwork, maplab, DLIO We really appreciate these open source projects!

Timeline

Early Release (2024-07-10)
Beta Release (TODO)

Related Papers

If our work has helped you, please cite:

@misc{he2024accuratepriorcentricmonocularpositioning,
      title={Accurate Prior-centric Monocular Positioning with Offline LiDAR Fusion}, 
      author={Jinhao He and Huaiyang Huang and Shuyang Zhang and Jianhao Jiao and Chengju Liu and Ming Liu},
      year={2024},
      eprint={2407.09091},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2407.09091}, 
}

liubonan123/APMP