/KP3D

Code for "Self-Supervised 3D Keypoint Learning for Ego-motion Estimation"

Self-Supervised 3D Keypoint Learning for Ego-motion Estimation

Code coming soon.

Overview

  • Sparse mono-SfM: A new framework for the simultaneous learning of keypoint detection, matching and 3D lifting by incorporating a differentiable pose estimation module.
  • Multi-view adaptation: A novel adaptation technique that exploits the temporal context in videos to further boost the repeatability and matching performance of the keypoint network.
  • State-of-the-art performance: We integrate the networks into a visual odometry framework, enabling robust and accurate ego-motion estimation results

[Full paper] [YouTube]

Trajectories

Trajectories of DS-DSO on KITTI odometry sequences 00-10: ds_dso_kitti_00_10.zip. We also include the results of our ablative analysis as well as our evaluation of monodepth2.

trajectory seq05 trajectory seq07

Citation

Please use the following citation when referencing our work:

@misc{selfsupkp3d2019,
  title = {{Self-Supervised 3D Keypoint Learning for Ego-motion Estimation}},
  author = {Jiexiong Tang and Rares Ambrus and Vitor Guizilini and Sudeep Pillai and Hanme Kim and Adrien Gaidon},
    year={2019},
    eprint={1912.03426},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}