/awesome-point-cloud-registration

A curated list of point cloud registration.

awesome-point-cloud-registration

A curated list of resources on point cloud registration inspired by awesome-computer-vision. Work-in-progress. All contributions are welcome and appreciated.

This list focuses on the rigid registration between point clouds.

Table of Contents

Coarse Registration

The coarse registration methods (or global registration) aligns two point clouds without an initial guess. We broadly classified these methods into feature matching based, end-to-end, randomized and probabilistic. Most of the learning based methods are focusing on some specific step in the feature matching based algorithms.

Feature Matching Based

The feature-matching based registration algorithms generally follow a two-stage workflow: determining correspondence and estimate the transformation. The correspondence establishing stage usually follow the four-step pipeline: keypoint detection, feature description, descriptor matching and outlier rejection. The nearest neighbor matching is the de-facto matching strategy, but could be replaced by learnable matching stategies. We also include some papers which adopt the graph algorithms for the matching and outlier rejection problem.

Keypoint Detection

  • HKS: A Concise and Provably Informative Multi‐Scale Signature Based on Heat Diffusion. CGF'2009 [paper]
  • Harris3D: a robust extension of the harris operator for interest point detection on 3D meshes. VC'2011 [paper]
  • Intrinsic shape signatures: A shape descriptor for 3D object recognition. ICCV'2012 [paper]
  • Learning a Descriptor-Specific 3D Keypoint Detector. ICCV'2015 [paper]
  • 3DFeat-Net: Weakly Supervised Local 3D Features for Point Cloud Registration. ECCV'2018 [paper] [code]
  • USIP: Unsupervised Stable Interest Point Detection from 3D Point Clouds. ICCV'2019 [paper] [code]
  • D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features. CVPR'2020 [paper] [code]
  • PointCloud Saliency Maps. ICCV'2019 [paper] [code]
  • SK-Net: Deep Learning on Point Cloud via End-to-end Discovery of Spatial Keypoints. AAAI'2020 [paper]
  • SKD: Unsupervised Keypoint Detecting for Point Clouds using Embedded Saliency Estimation. arxiv'2019 [paper]
  • Fuzzy Logic and Histogram of Normal Orientation-based 3D Keypoint Detection For Point Clouds. PRL'2020 [paper]
  • MaskNet: A Fully-Convolutional Network to Estimate Inlier Points. 3DV'2020 [paper] [code]
  • PREDATOR: Registration of 3D Point Clouds with Low Overlap. arxiv'2020 [paper] [code]

Survey:

  • Performance Evaluation of 3D Keypoint Detectors. IJCV'2013 [paper]

Feature Description

  • Spin Image: Using spin images for efficient object recognition in cluttered 3D scenes. TPAMI'1999 [paper]
  • USC: Unique shape context for 3D data description. 3DOR'2010 [paper]
  • 3DShapeContext: Recognizing Objects in Range Data Using Regional Point Descriptors. ECCV'2004 [paper]
  • SHOT: Unique Signatures of Histograms for Local Surface Description. ECCV'2010 [paper]
  • FPFH: Fast Point Feature Histograms (FPFH) for 3D registration. ICRA'2009 [paper]
  • RoPS: 3D Free Form Object Recognition using Rotational Projection Statistics. WACV'2013 [paper]
  • CGF: Learning Compact Geometric Features. ICCV'2017 [paper]
  • 3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions. CVPR'2017 [paper] [code]
  • End-to-end learning of keypoint detector and descriptor for pose invariant 3D matching. CVPR'2018 [paper]
  • PPFNet: Global Context Aware Local Features for Robust 3D Point Matching. CVPR'2018 [paper]
  • 3DFeat-Net: Weakly Supervised Local 3D Features for Point Cloud Registration. ECCV'2018 [paper] [code]
  • MVDesc: Learning and Matching Multi-View Descriptors for Registration of Point Clouds. ECCV'2018 [paper] [code]
  • FoldingNet: Point Cloud Auto-encoder via Deep Grid Deformation. CVPR'2018 [paper] [code]
  • PPF-FoldNet: Unsupervised Learning of Rotation Invariant 3D Local Descriptors. ECCV'2018 [paper] [code]
  • 3D Local Features for Direct Pairwise Registration. CVPR'2019 [paper]
  • 3D Point-Capsule Networks. CVPR'2019 [paper] [code]
  • The Perfect Match: 3D Point Cloud Matching with Smoothed Densities. CVPR'2019 [paper] [code]
  • FCGF: Fully Convolutional Geometric Features. ICCV'2019 [paper] [code]
  • Learning an Effective Equivariant 3D Descriptor Without Supervision. ICCV'2019 [paper]
  • D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features. CVPR'2020 [paper] [code]
  • End-to-End Learning Local Multi-view Descriptors for 3D Point Clouds. CVPR'2020 [paper] [code]
  • LRF-Net- Learning Local Reference Frames for 3D Local Shape Description and Matching. arxiv'2020 [paper]
  • DH3D: Deep Hierarchical 3D Descriptors for Robust Large-Scale 6DoF Relocalization. ECCV'2020 [paper] [code]
  • Distinctive 3D local deep descriptors. arxiv'2020 [paper] [code]
  • SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration. CVPR'2021 [paper] [code]
  • PREDATOR: Registration of 3D Point Clouds with Low Overlap. CVPR'2021 [paper] [code]
  • Self-supervised Geometric Perception. CVPR'2021 [paper] [code]
  • 3D Point Cloud Registration with Multi-Scale Architecture and Self-supervised Fine-tuning. arxiv'2021 [paper] [code]
  • Generalisable and Distinctive (GeDi) 3D local deep descriptors for point cloud registration. arxiv'2021 [paper] [code]
  • Neighborhood Normalization for Robust Geometric Feature Learning. CVPR'2021 [paper] [code]
  • UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Rendering. CVPR'2021 [paper] [code]
  • Bootstrap Your Own Correspondences. ICCV'2021 [paper] [code]
  • UPDesc: Unsupervised Point Descriptor Learning for Robust Registration. arxiv'2021 [paper]
  • You Only Hypothesize Once: Point Cloud Registration with Rotation-equivariant Descriptors. ICCV'2021 [paper] [code]
  • P2-Net: Joint Description and Detection of Local Features for Pixel and Point Matching. ICCV'2021 [paper]
  • Distinctiveness oriented Positional Equilibrium for Point Cloud Registration. ICCV'2021 [paper]
  • CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration. NeurIPS'2021 [paper] [code]
  • IMFNet: Interpretable Multimodal Fusion for Point Cloud Registration. arxiv'2021 [paper]
  • Lepard: Learning partial point cloud matching in rigid and deformable scenes. arxiv'2021 [paper] [code]
  • Fast and Robust Registration of Partially Overlapping Point Clouds. RA-L'2021 [paper] [code]
  • Geometric Transformer for Fast and Robust Point Cloud Registration. arxiv'2022 [paper] [code]

Survey:

  • A Comprehensive Performance Evaluation of 3D Local Feature Descriptors. IJCV'2015 [paper]
  • Evaluating Local Geometric Feature Representations for 3D Rigid Data Matching. ICIP'2019 [paper]

Outlier Rejection

We also include the algorithms designed for finding matching between keypoints given descriptors (which replaces nearest-neighbor-searching) in this section.

  • RANSAC: Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. 1981 [paper]
  • Locally Optimized RANSAC. 2003 [paper]
  • Graph-cut RANSAC. CVPR'2018 [paper] [code]
  • MAGSAC: Marginalizing Sample Consensus. CVPR'2019 [paper] [code]
  • VFC: A Robust Method for Vector Field Learning with Application To Mismatch Removing. CVPR'2011 [paper]
  • In Search of Inliers: 3D Correspondence by Local and Global Voting. CVPR'2014 [paper]
  • FGR: Fast Global Registration. ECCV'2016 [paper] [code]
  • Ranking 3D Feature Correspondences Via Consistency Voting. PRL'2019 [paper]
  • An Accurate and Efficient Voting Scheme for a Maximally All-Inlier 3D Correspondence Set. TPAMI'2020 [paper]
  • GORE: Guaranteed Outlier Removal for Point Cloud Registration with Correspondences. TPAMI'2018 [paper] [code]
  • A Polynomial-time Solution for Robust Registration with Extreme Outlier Rates. RSS'2019 [paper]
  • Graduated Non-Convexity for Robust Spatial Perception: From Non-Minimal Solvers to Global Outlier Rejection. ICRA'2020 [paper]
  • TEASER: Fast and Certifiable Point Cloud Registration. T-RO'2020 [paper] [code]
  • One Ring to Rule Them All: Certifiably Robust Geometric Perception with Outliers. NeurIPS'2020 [paper]
  • SDRSAC: Semidefinite-Based Randomized Approach for Robust Point Cloud Registration without Correspondences. CVPR'2019 [paper] [code]
  • Robust Low-Overlap 3D Point Cloud Registration for Outlier Rejection. ICRA'2019 [paper]
  • ICOS: Efficient and Highly Robust Rotation Search and Point Cloud Registration with Correspondences. arxiv'2021 [paper]
  • Fast Semantic-Assisted Outlier Removal for Large-scale Point Cloud Registration. arxiv'2022 [paper]

Learning based (including 2D outlier rejection methods)

  • Learning to Find Good Correspondences. CVPR'2018 [paper] [code]
  • NM-Net: Mining Reliable Neighbors for Robust Feature Correspondences. CVPR'2019 [paper] [code]
  • OANet: Learning Two-View Correspondences and Geometry using Order-Aware Network. ICCV'2019 [paper] [code]
  • ACNe: Attentive Context Normalization for Robust Permutation-Equivariant Learning. CVPR'2020 [paper] [code]
  • SuperGlue: Learning Feature Matching with Graph Neural Networks. CVPR'2020 [paper] [code]
  • 3DRegNet: A Deep Neural Network for 3D Point Registration. CVPR'2020 [paper] [code]
  • Deep Global Registration. CVPR'2020 [paper] [code]
  • 3D Correspondence Grouping with Compatibility Features. arxiv'2020 [paper]
  • PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency. CVPR'2021 [paper] [code]
  • StickyPillars: Robust and Efficient Feature Matching on Point Clouds using Graph Neural Networks. CVPR'2021 [paper]
  • HRegNet: A Hierarchical Network for Large-scale Outdoor LiDAR Point Cloud Registration. ICCV'2021 [paper] [code]
  • Keypoint Matching for Point Cloud Registration using Multiplex Dynamic Graph Attention Networks. RA-L'2021 [paper]
  • Deep Hough Voting for Robust Global Registration. ICCV'2021 [paper]
  • End-to-end Learning the Partial Permutation Matrix for Robust 3D Point Cloud Registration. arxiv'2021 [paper]
  • TriVoC: Efficient Voting-based Consensus Maximization for Robust Point Cloud Registration with Extreme Outlier Ratios. arxiv'2021 [paper]
  • COTReg: Coupled Optimal Transport based Point Cloud Registration. arxiv'2021 [paper]
  • DetarNet: Decoupling Translation and Rotation by Siamese Network for Point Cloud Registration. AAAI'2022 [paper] [code]

Survey

  • A Performance Evaluation of Correspondence Grouping Methods for 3D Rigid Data Matching. TPAMI'2019 [paper]

Graph Algorithms

  • A Graduated Assignment Algorithm for Graph Matching. TPAMI'1996 [paper]
  • A Spectral Technique for Correspondence Problems Using Pairwise Constraints. ICCV'2005 [paper] [code]
  • Balanced Graph Matching. NIPS'2006 [paper] [code]
  • Feature Correspondence via Graph Matching: Models and Global Optimization. ECCV'2008 [paper]
  • An Integer Projected Fixed Point Method for Graph Matching and MAP Inference. NIPS'2009 [paper] [code]
  • Optimal Correspondences From Pairwise Constraints. ICCV'2009 [paper]
  • Reweighted Random Walks for Graph Matching. ECCV'2010 [paper]
  • Maximal Cliques Based Rigid Body Motion Segmentation with a RGB-D Camera. ACCV'2012 [paper]
  • A Probabilistic Approach to Spectral Graph Matching. TPAMI'2013 [paper]
  • A Practical Maximum Clique Algorithm for Matching with Pairwise Constraints. arxiv'2019 [paper]
  • ROBIN: a Graph-Theoretic Approach to Reject Outliers in Robust Estimation using Invariants. arxiv'2020 [paper]
  • CLIPPER A Graph-Theoretic Framework for Robust Data Association. arxiv'2020 [paper]
  • PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency CVPR'2021 [paper] [code]
  • Pairwise Point Cloud Registration Using Graph Matching and Rotation-invariant Features. arxiv'2021 [paper]

End-to-End

Some papers perform end-to-end registration by directly predicting a rigid transformation aligning two point clouds without explicitly following the detection -- description -- outlier filtering pipepline. While they works well on object-centric datasets, the performance on real-world scene registration is not satisfactory.

  • PointNetLK: Robust & Efficient Point Cloud Registration using PointNet. CVPR'2019 [paper] [code]
  • Deep Closest Point: Learning Representations for Point Cloud Registration. ICCV'2019 [paper] [code]
  • PRNet: Self-Supervised Learning for Partial-to-Partial Registration. NeurIPS'2019 [paper] [code]
  • AlignNet-3D: Fast Point Cloud Registration of Partially Observed Objects. 3DV'2019 [paper] [code]
  • RPM-Net: Robust Point Matching using Learned Features. CVPR'2020 [paper] [code]
  • Feature-metric Registration: A Fast Semi-supervised Approach for Robust Point Cloud Registration without Correspondences. CVPR'2020 [code] [code]
  • Iterative Distance-Aware Similarity Matrix Convolution with Mutual-Supervised Point Elimination for Efficient Point Cloud Registration. ECCV'2020 [paper] [code]
  • Self-supervised Point Set Local Descriptors for Point Cloud Registration. arxiv'2020 [paper]
  • Learning 3D-3D Correspondences for One-shot Partial-to-partial Registration. arxiv'2020 [paper]
  • Robust Point Cloud Registration Framework Based on Deep Graph Matching. CVPR'2021 [paper] [code]
  • RPSRNet: End-to-End Trainable Rigid Point Set Registration Network using Barnes-Hut 2^D-Tree Representation. CVPR'2021 [paper]
  • Deep Weighted Consensus (DWC) Dense correspondence confidence maps for 3D shape registration. arxiv'2021 [paper]
  • OMNet: Learning Overlapping Mask for Partial-to-Partial Point Cloud Registration. arxiv'2021 [paper]
  • FINet: Dual Branches Feature Interaction for Partial-to-Partial Point Cloud Registration. arxiv'2021 [paper]
  • PointNetLK Revisited. CVPR'2021 [paper]
  • Point Cloud Registration using Representative Overlapping Points. arxiv'2021 [paper] [code]
  • Correspondence-Free Point Cloud Registration with SO(3)-Equivariant Implicit Shape Representations. arxiv'2021 [paper]
  • Geometry Guided Network for Point Cloud Registration. RA-L'2021 [paper]
  • (Just) A Spoonful of Refinements Helps the Registration Error Go Down. ICCV'2021 [paper] [code]
  • A Robust Loss for Point Cloud Registration. ICCV'2021 [paper]
  • Sampling Network Guided Cross-Entropy Method for Unsupervised Point Cloud Registration. ICCV'2021 [paper] [code]
  • DeepBBS: Deep Best Buddies for Point Cloud Registration. 3DV'2021 [paper] [code]
  • PCAM: Product of Cross-Attention Matrices for Rigid Registration of Point Clouds. ICCV'2021 [paper] [code]
  • Feature Interactive Representation for Point Cloud Registration. ICCV'2021 [paper]
  • DeepPRO: Deep Partial Point Cloud Registration of Objects. ICCV'2021 [paper]
  • What Stops Learning-based 3D Registration from Working in the Real World? arxiv'2021 [paper]
  • Full Transformer Framework for Robust Point Cloud Registration with Deep Information Interaction. arxiv'2021 [paper] [code]

Randomized

  • RANSAC: Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. 1981 [paper]
  • 4PCS: 4-points Congruent Sets for Robust Pairwise Surface Registration. TOG'2008 [paper]
  • Model Globally, Match Locally: Efficient and Robust 3D Object Recognition. CVPR'2010 [paper] [code]
  • Super 4PCS: Fast Global Pointcloud Registration via Smart Indexing. CGF'2014 [paper] [code]

Probabilistic

  • Point Set Registration: Coherent Point Drift. TPAMI'2010 [paper] [code]
  • Robust Point Set Registration Using Gaussian Mixture Models. TPAMI'2011 [paper] [code]
  • A Generative Model for the Joint Registration of Multiple Point Sets. ECCV'2014 [paper]
  • Aligning the Dissimilar: A Probabilistic Method for Feature-based Point Set Registration. ICPR'2016 [paper]
  • A Probabilistic Framework for Color-Based Point Set Registration. CVPR'2016 [paper]
  • Density Adaptive Point Set Registration. CVPR'2018 [paper] [code]
  • HGMR: Hierarchical Gaussian Mixtures for Adaptive 3D Registration. ECCV'2018 [paper]
  • Robust Feature-Based Point Registration Using Directional Mixture Model. arxiv'2019 [paper]
  • FilterReg: Robust and Efficient Probabilistic Point-Set Registration using Gaussian Filter and Twist Parameterization. CVPR'2019 [paper] [code]
  • PointGMM: a Neural GMM Network for Point Clouds. CVPR'2020 [paper] [code]
  • DeepGMR: Learning Latent Gaussian Mixture Models for Registration. ECCV'2020 [paper] [code]
  • Registration Loss Learning for Deep Probabilistic Point Set Registration. 3DV'2020 [paper] [code]
  • A Termination Criterion for Probabilistic PointClouds Registration. arxiv'2020 [paper]
  • LSG-CPD: Coherent Point Drift with Local Surface Geometry for Point Cloud Registration. ICCV'2021 [paper]

Others

  • Gravitational Approach for Point Set Registration. CVPR'2016 [paper]
  • Accelerated Gravitational Point Set Alignment with Altered Physical Laws. ICCV'2019 [paper]
  • Fast Gravitational Approach for Rigid Point Set Registration with Ordinary Differential Equations. arxiv'2020 [paper]
  • Minimal Solvers for Mini-Loop Closures in 3D Multi-Scan Alignment. CVPR'2019 [paper]
  • Minimal Solvers for 3D Scan Alignment With Pairs of Intersecting Lines. CVPR'2020 [paper]
  • Learning multiview 3D point cloud registration. CVPR'2020 [paper] [code]
  • A Dynamical Perspective on Point Cloud Registration. arxiv'2020 [paper]
  • Plane Pair Matching for Efficient 3D View Registration. arxiv'2020 [paper]
  • On Bundle Adjustment for Multiview Point Cloud Registration. RA-L'2021 [paper]
  • Provably Approximated Point Cloud Registration. ICCV'2021 [paper]
  • GenReg: Deep Generative Method for Fast Point Cloud Registration. arxiv'2021 [paper]
  • Multiway Non-rigid Point Cloud Registration via Learned Functional Map Synchronization. arxiv'2021 [paper]
  • DeepUME: Learning the Universal Manifold Embedding for Robust Point Cloud Registration. BMVC'2021 [paper]

Fine Registration

The fine registration methods (or local registration) produce highly precise registration results, given the initial pose between two point clouds.

Traditional

  • Point2Point ICP: A Method for Registration of 3-D Shapes. TPAMI'1992 [paper]
  • Point2Plane Object Modelling by Registration of Multiple Range Images. TPAMI'1992 [paper]
  • RPM: New Algorithms for 2D and 3D Point Matching: Pose Estimation and Correspondence. [paper]
  • Matching of 3-D Curves using Semi-differential Invariants. ICCV'1995 [paper]
  • The Trimmed Iterative Closest Point Algorithm. 2002 [paper]
  • Comparing ICP Variants on Real-World Data Sets. 2013 [paper]
  • Generalized-ICP. RSS'2009 [paper] [code]
  • Go-ICP: Solving 3D Registration Efficiently and Globally Optimally. ICCV'2013 [paper] [code]
  • AA-ICP: Iterative Closest point with Anderson Acceleration. ICRA'2018 [paper]
  • Point Clouds Registration with Probabilistic Data Association. IROS'2016 [paper] [code]
  • GH-ICP：Iterative Closest Point Algorithm with Global Optimal Matching and Hybrid Metric. 3DV'2018 [paper] [code]
  • NDT: The Normal Distributions Transform: A New Approach To Laser Scan Matching. IROS'2003 [paper]
  • Best Buddies Registration for Point Clouds. ACCV'2020 [paper]
  • Provably Approximated ICP. arxiv'2021 [paper]

Learning-based

  • DeepICP: An End-to-End Deep Neural Network for 3D Point Cloud Registration. ICCV'2019 [paper]

Datasets

Tools