/awesome-visual-localization

A curated list of awesome visual localization research works.

Awesome Visual Localization

A curated list of visual (re)localization resources, inspired by awesome-computer-vision. The list focuses on the research of visual localization, i.e. estimates 6 DoF camera pose of a query RGB/RGB-D frame in known scenes.

This document is a work in progress. Please suggest papers/resources through pull requests. If you believe this list is missing something or has factually inaccurate info, you can also new an issue. All contributions are appreciated.

Table of Contents

Direct Approaches

Approaches of this type estimate the camera pose directly from the query frame.

Image Retrieval

  • [2018 CVPR] PointNetVLAD: Deep Point Cloud Based Retrieval for Large-Scale Place Recognition [paper] [code]
  • [2017 CVPR] Are Large-Scale 3D Models Really Necessary for Accurate Visual Localization? [paper]
  • [2016 CVPR] NetVLAD: CNN architecture for weakly supervised place recognition [paper] [code]
  • [2016 CVPR] Large-Scale Location Recognition and the Geometric Burstiness Problem [paper] [code]
  • [2015 CVPR] 24/7 place recognition by view synthesis [paper] [code]
  • [2014 TVCG] Real-Time RGB-D Camera Relocalization via Randomized Ferns for Keyframe Encoding [paper]
  • [2013 CVPR] All about VLAD [paper]
  • [2012 BMVC] Image Retrieval for Image-Based Localization Revisited [paper]
  • [2012 BMVC] 6D Relocalisation for RGBD Cameras Using Synthetic View Regression
  • [2011 IROS] Real-Time Loop Detection with Bags of Binary Words [paper]

Pose Regression

  • [2020 ECCV] 6D Camera Relocalization in Ambiguous Scenes via Continuous Multimodal Inference [paper] [code]
  • [2020 AAAI] AtLoc: Attention Guided Camera Localization [paper] [code]
  • [2019 CVPR] Understanding the Limitations of CNN-based Absolute Camera Pose Regression [paper]
  • [2018 ECCV] RelocNet: Continuous Metric Learning Relocalisation using Neural Nets [paper]
  • [2018 BMVC] Improved Visual Relocalization by Discovering Anchor Points [paper] [code]
  • [2018 CVPR] Geometry-Aware Learning of Maps for Camera Localization [paper] [code]
  • [2017 ICCV Workshop] Camera Relocalization by Computing Pairwise Relative Poses Using Convolutional Neural Network [paper] [code]
  • [2017 CVPR] Geometric loss functions for camera pose regression with deep learning [paper]
  • [2017 CVPR] Image-based localization using LSTMs for structured feature correlation [paper]
  • [2016 ICRA] Modelling Uncertainty in Deep Learning for Camera Relocalization [paper]
  • [2015 ICCV] PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization [paper]

Two-Step Approaches

Approaches of this type first find correspondences between the query frame and the scene, then estimate the camera pose through optimization with RANSAC.

Feature Matching Based

  • [2021 ICCV Workshop] MegLoc: A Robust and Accurate Visual Localization Pipeline [paper]
  • [2021 CVPR] Back to the Feature: Learning Robust Camera Localization from Pixels to Pose [paper] [code]
  • [2020 ECCV Workshop] Hierarchical Localization with hloc and SuperGlue [slides] [code]
  • [2019 CVPR] From Coarse to Fine: Robust Hierarchical Localization at Large Scale [paper] [code]
  • [2019 CVPR] D2-Net: A Trainable CNN for Joint Description and Detection of Local Features [paper] [code]
  • [2018 CoRL] Leveraging Deep Visual Descriptors for Hierarchical Efficient Localization [paper] [code]
  • [2018 CVPR] InLoc: Indoor Visual Localization with Dense Matching and View Synthesis [paper] [code]
  • [2017 ICCV] Efficient Global 2D-3D Matching for Camera Localization in a Large-Scale 3D Map [paper]
  • [2017 ICRA] [RA-L] Self-supervised Visual Descriptor Learning for Dense Correspondence [paper]
  • [2016 TPAMI] Efficient & Effective Prioritized Matching for Large-Scale Image-Based Localization [paper]
  • [2015 ICCV] Camera Pose Voting for Large-Scale Image-Based Localization [paper]
  • [2012 ECCV] Improving Image-Based Localization by Active Correspondence Search [paper] [code]
  • [2012 CVPR] Real-time Image-based 6-DOF Localization in Large-Scale Environments [paper]
  • [2011 ICCV] Fast Image-Based Localization using Direct 2D-to-3D Matching [paper]

Scene Coordinate Regression

  • [2022 CVPR] CrossLoc: Scalable Aerial Localization Assisted by Multimodal Synthetic Data [paper] [code]
  • [2021 ICCV] Continual Learning for Image-Based Camera Localization [paper] [code]
  • [2021 TPAMI] Visual Camera Re-Localization from RGB and RGB-D Images Using DSAC [paper] [code]
  • [2021 CVPR] Robust Neural Routing Through Space Partitions for Camera Relocalization in Dynamic Indoor Environments [paper] [code]
  • [2021 CVPR] Learning Camera Localization via Dense Scene Matching [paper] [code]
  • [2021 CVPR] VS-Net: Voting with Segmentation for Visual Localization [paper] [code]
  • [2020 CVPR] KFNet: Learning Temporal Camera Relocalization using Kalman Filtering [paper] [code]
  • [2020 CVPR] Hierarchical Scene Coordinate Classification and Regression for Visual Localization [paper] [code]
  • [2019 TPAMI] Real-Time RGB-D Camera Pose Estimation in Novel Scenes using a Relocalisation Cascade [paper] [code]
  • [2019 ICCV] SANet: Scene Agnostic Network for Camera Localization [paper] [code]
  • [2019 ICCV] Expert Sample Consensus Applied to Camera Re-Localization [paper] [code]
  • [2018 IROS] Exploiting Points and Lines in Regression Forests for RGB-D Camera Relocalization [paper]
  • [2018 RSS] Full-Frame Scene Coordinate Regression for Image-Based Localization [paper]
  • [2018 CVPR] Learning Less is More – 6D Camera Localization via 3D Surface Regression [paper] [code]
  • [2017 IROS] Backtracking Regression Forests for Accurate Camera Relocalization [paper]
  • [2017 CVPR] DSAC - Differentiable RANSAC for Camera Localization [paper] [code]
  • [2017 CVPR] On-the-Fly Adaptation of Regression Forests for Online Camera Relocalisation [paper] [code]
  • [2017 ICRA] Random Forests versus Neural Networks − What’s Best for Camera Localization? [paper]
  • [2016 CVPR] Uncertainty-Driven 6D Pose Estimation of Objects and Scenes from a Single RGB Image [paper]
  • [2015 CVPR] Exploiting Uncertainty in Regression Forests for Accurate Camera Relocalization [paper]
  • [2014 CVPR] Multi-Output Learning for Camera Relocalization [paper]
  • [2013 CVPR] Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images [paper]

Datasets and Benchmarks

Indoor

Outdoor

Challenges

Tools