/awesome-image-alignment-and-stitching

A curated list of awesome resources for image alignment and stitching ...

awesome-image-alignment-and-stitching

A curated list of awesome resources for image alignment and stitching, etc. The list will be updated continuously.

❗ An up-to-date paper list can be found here.

Tutorials

  • Repo: Image registration

    A resource list about image registration related to natural/remote sensing/medical image and point cloud.

  • MICCAI2019 Tutorial: Learn2Reg: Tutorial on Deep Learning in Medical Image Registration [Homepage]

    Three hands-on sessions guiding participants to understand and implement published algorithms using clinical imaging data, including unsupervised registration, label-supervised registration, and discrete deep learning registration.

  • A Practical Review on Medical Image Registration: from Rigid to Deep Learning based Approaches [PDF&Slides]

    A tutorial for anyone who wants to learn Medical Image Registration, by Natan Andrade, Fabio Augusto Faria, Fábio Augusto Menocci Cappabianco, SIBGRAPI2018

  • [Blog] Image Registration: From SIFT to Deep Learning [Homepage] [Notes/Chinese]

    It presents OpenCV feature-based methods before diving into Deep Learning, including example codes.

  • [Book] Image Descriptor: Modern Approaches, Bin Fan, Zhenhua Wang, and Fuchao Wang. Springer, 2015.

  • [Report] 刘帅成：图像对齐技术及其应用 [Video]

  • OpenCV Stitching Tutorial

Review

Matching/Registration/Alignment

  • [2017CVPR] HPatches: A Benchmark and Evaluation of Handcrafted and Learned Local Descriptors [Homepage]

Stitching/Mosaicking

  • [2019GRSM] Remote Sensing Image Mosaicking: Achievements and Challenges
  • [2019IVC] Image mosaicing: A deeper insight
  • [ 2006Szeliski] Image Alignment and Stitching: A Tutorial

Remote Sensing

  • [2018ISPRSJ] A deep learning framework for remote sensing image registration

Medical Registration

  • Deep Learning in Medical Image Registration: A Survey, 2019
  • Learning image-based spatial transformations via convolutional neural networks: A review, Magnetic Resonance Imaging, December 2019
  • Deformable Medical Image Registration: A Survey, TMI2013

Geometric Pre-processing

  • [2019TCSVT] DR-GAN: Automatic Radial Distortion Rectification Using Conditional GAN in Real-Time
  • [2019CVPR] Learning to Calibrate Straight Lines for Fisheye Image Rectification
  • [2019CVPR] Deep Single Image Camera Calibration with Radial Distortion [Proj]
  • [2015CVPR] Line-Based Multi-Label Energy Optimization for Fisheye Image Rectification and Calibration [Proj]
  • [2019IJCV] Superpixel-Guided Two-View Deterministic Geometric Model Fitting [Guobao Xiao]

Image Matching

Feature Detection

- Key Point

  • GLAMpoints: Greedily Learned Accurate Match points, arXiv2019.8

  • [2018NeurIPS] LF-Net: Learning Local Features from Images [github]

  • [2017CVPR] Comparative Evaluation of Hand-Crafted and Learned Local Features

  • [2016ECCV] LIFT: Learned Invariant Feature Transform [Theano] [TF] [Eduard Trulls]

- Line Detection

  • Line as object: datasets and framework for semantic line segment detection, arXiv2019.9

  • [2020PAMI] Learning Regional Attraction for Line Segment Detection

  • [2019CVPR] Learning Attraction Field Representation for Robust Line Segment Detection [github]

  • [2019CVPR] PPGNet: Learning Point-Pair Graph for Line Segment Detection [github]

  • [2018CVPR] [Wireframe] Learning to Parse Wireframes in Images of Man-Made Environments [github]

  • [2017ICCV] Semantic Line Detection and Its Applications

  • [2017PAMI] A Novel Linelet-based Representation for Line Segment Detection [Code]

  • [2017CVPR] MCMLSD: A Dynamic Programming Approach to Line Segment Detection [Code]

  • Edge Drawing/EDLines/LBD Descriptors/LSD [Code] [Code2]

- Edge Detection

  • Semantic Edge Detection with Diverse Deep Supervision, arXiv2018.12
  • [2017CVPR] CASENet: Deep Category-Aware Semantic Edge Detection

- Ellipse detection

  • Dense Extreme Inception Network: Towards a Robust CNN Model for Edge Detection, arXiv2019.9

  • [2018TIP] Arc-support Line Segments Revisited: An Efficient and High-quality Ellipse Detection [github]

  • [2017IJCV] Holistically-Nested Edge Detection

  • [2017TIP] A Fast Ellipse Detector Using Projective Invariant Pruning [Code]

  • [2012ECCV] [ELSD] A parameterless line segment and elliptical arc detector with enhanced ellipse fitting [Homepage&Code]

Feature Description

  • Single and Cross-Dimensional Feature Detection and Description: An Evaluation, arXiv2019.10

  • [2020AAAI] LCD: Learned Cross-domain Descriptors for 2D-3D Matching

  • [2019ICCV] Beyond Cartesian Representations for Local Descriptors

  • [2016CVPR] Accumulated Stability Voting: A Robust Descriptor from Descriptors of Multiple Scales [Proj] [Code]

  • [2015ICCV] Discriminative learning of deep convolutional feature point descriptors

Image Matching

- Feature Matching

  • GLMNet: Graph Learning-Matching Networks for Feature Matching, arXiv2019.11

  • SuperGlue: Learning Feature Matching with Graph Neural Networks, arXiv2019.11

  • [2019IJCV] Locality Preserving Matching

  • [2019TIP] Robust Feature Matching Using Spatial Clustering With Heavy Outliers

  • [2019CVPR] RF-Net: An End-to-End Image Matching Network based on Receptive Field [Code]

  • [2019ICCV] Learning Two-View Correspondences and Geometry Using Order-Aware Network [Codes]

  • [2018PAMI] CODE: Coherence based Decision Boundaries for Feature Matching [Code]

  • [2018PAMI] Best-buddies similarity-robust template matching using mutual nearest neighbors

    [2015CVPR] Best-Buddies Similarity for Robust Template Matching [Proj&Code]

  • [2018CVIU] Deep Spectral Correspondence for Matching Disparate Image Pairs

  • [2017MM] Deep Matching and Validation Network: An End-to-End Solution to Constrained Image Splicing Localization and Detection [Codes]

  • [2017ICMR] Panorama to panorama matching for location recognition

  • [2016IJCV] DeepMatching: Hierarchical Deformable Dense Matching [Code]

  • [2016BMVC] Fully-Trainable Deep Matching

  • [2016NeurIPS] Universal Correspondence Network

  • [2015CVPR] MatchNet: Unifying Feature and Metric Learning for Patch-Based Matching

  • [2013CVPR] Joint Spectral Correspondence for Disparate Image Matching [Code]

  • [2013HPCS] Gpu-asift: A fast fully affine-invariant feature extraction algorithm [Homepage] [Code]

  • [2012CVPR] Aligning Images in the Wild [Homepage]

  • [2012CVPR] Image Matching using Local Symmetry Features [Proj&Code]

- Line Matching

  • [2012PR] [LPI] Robust Line Matching through Line-point Invariants, [Code]

    [2010CVPR] Line Matching Leveraged By Point Correspondences [Proj]

- OutlierRemoval

  • [2019ICCV] Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses [github] [Proj]

  • [2019CVPR] MAGSAC: marginalizing sample consensus (no in-outlier threshold) [Code]

  • [2018CVPR] Latent RANSAC [Code]

  • [2017CVPR] GMS: Grid-based Motion Statistics for Fast, Ultra-robust Feature Correspondence [Code] [Talk]

- Cross-View Matching

  • [2019ICCV] Bridging the Domain Gap for Ground-to-Aerial Image Matching [Code]
  • [2019ICCV] Ground-to-Aerial Image Geo-Localization With a Hard Exemplar Reweighting Triplet Loss
  • [2018CVPR] CVM-Net: Cross-View Matching Network for Image-Based Ground-to-Aerial Geo-Localization
  • [2017CVPR] Cross-View Image Matching for Geo-Localization in Urban Environments
  • [2016CVPR] Learning to Match Aerial Images With Deep Attentive Architectures
  • [2016CVPR] Regularity-Driven Facade Matching Between Aerial and Street Views
  • [2015ICCV] Wide-Area Image Geolocalization With Aerial Reference Imagery
  • [2015CVPR] Learning Deep Representations for Ground-to-Aerial Geolocalization
  • [2013CVPR] Cross-View Image Geolocalization

Image Registration

  • Report: Deep Learning: Image Registration, Steven Chen and Ty Nguyen, 2017 [Slides]

Homography

  • Homography-Estimation-List_pwc

  • Content-Aware Unsupervised Deep Homography Estimation, arXiv2019.9 [Code]

  • [2019Acce] Combining Convolutional Neural Network and Photometric Refinement for Accurate Homography Estimation

  • [2019ICCV] Homography from two orientation- and scale-covariant features [Code]

  • [2019AppSci] STN-Homography: Direct Estimation of Homography Parameters for Image Pairs, AppliedScience

  • [2018ACCV] Rethinking Planar Homography Estimation Using Perspective Fields [Code]

  • [2018RAL] Unsupervised Deep Homography: A Fast and Robust Homography Estimation Model, IEEE RAL [Code]

  • [2018ECCV] GridFace: Face Rectification via Learning Local Homography Transformations

  • [2017CVPR] CLKN: Cascaded Lucas-Kanade Networks for Image Alignment

  • [2017ICCVW] Homography Estimation from Image Pairs with Hierarchical Convolutional Networks [Code]

  • [2016RSSW] [HomographyNet] Deep Image Homography Estimation, RSSW2016 [Note] [Note2] [unofficial1-tf] [unofficial2-tf] [unofficial3] [unofficial4-pytorch] [unofficial5-keras] [unofficial6-keras]

  • Auxiliary

  • Convolutional Neural Network Optimization for Homography Estimation, thesis2018

  • Homography Estimation using Deep Learning for Registering All-22 Football Video Frames, TechReport2017 [PDF]

Registration

- Natural Scene

  • [2017PAMI] Transformations Based on Continuous Piecewise-Affine Velocity Fields

    [2015ICCV] Highly-Expressive Spaces of Well-Behaved Transformations: Keeping It Simple [Code-Python+CUDA] [Code-CPU+Julia] [Proj]

  • [2015CVPR] Robust Image Alignment with Multiple Feature Descriptors and Matching-Guided Neighborhoods [Proj/Code]

  • [2014ECCV] Multi-modal and Multi-spectral Registration for Natural Images [Proj]

- Remote Sensing

  • Coarse to fine non-rigid registration: a chain of scale-specific neural networks for multimodal image alignment with application to remote sensing, arXiv2018.2

  • [2018ECCV] Multimodal Image Alignment Through a Multiscale Chain of Neural Networks with Application to Remote Sensing [Homepage] [Code]

  • [2018Acce] Multi-Temporal Remote Sensing Image Registration Using Deep Convolutional Features [github]

  • CVPR 2019

  • Metric Learning for Image Registration [github]

  • Networks for Joint Affine and Non-Parametric Image Registration

  • Multiview 2D/3D Rigid Registration via a Point-Of-Interest Network for Tracking and Triangulation

  • SDRSAC: Semi-definite-Based Randomized Approach for Robust Point Cloud Registration Without Correspondences [Codes]

  • 3D Local Features for Direct Pairwise Registration

  • PointNetLK: Robust & Efficient Point Cloud Registration Using PointNet [Codes]

  • FilterReg: Robust and Efficient Probabilistic Point-Set Registration Using Gaussian Filter and Twist Parameterization [Codes]

  • NeuIPS2019

  • Arbicon-Net: Arbitrary Continuous Geometric Transformation Networks for Image Registration, NeurIPS2019
    Jianchun Chen (New York University), Lingjing Wang (New York University), Xiang Li (New York University), Yi Fang (New York University)

  • Recurrent Registration Neural Networks for Deformable Image Registration, NeuIPS2019
    Robin Sandkühler, Simon Andermatt, ...

  • PRNet: Self-Supervised Learning for Partial-to-Partial Registration, NeuIPS2019
    Yue Wang (MIT) · Justin M Solomon (MIT)

  • Region-specific Diffeomorphic Metric Mapping, NeurIPS 2019

  • Automatic and Robust Skull Registration Based on Discrete Uniformization, ICCV2019 Junli Zhao, Xin Qi, Chengfeng Wen, Na Lei, Xianfeng Gu

  • Linearly Converging Quasi Branch and Bound Algorithms for Global Rigid Registration, ICCV2019 Nadav Dym, Shahar Ziv Kovalsky

  • Deep Closest Point: Learning Representations for Point Cloud Registration, ICCV2019 Yue Wang, Justin M. Solomon

  • A Deep Step Pattern Representation for Multimodal Retinal Image Registration, ICCV2019 Jimmy Addison Lee, Peng Liu, Jun Cheng, Huazhu Fu

  • Efficient and Robust Registration on the 3D Special Euclidean Group, ICCV2019 Uttaran Bhattacharya, Venu Madhav Govindu

  • Robust Variational Bayesian Point Set Registration, ICCV2019 Jie Zhou, Xinke Ma, Li Liang, Yang Yang, Shijin Xu, Yuhe Liu, Sim-Heng Ong

  • Recursive Cascaded Networks for Unsupervised Medical Image Registration, ICCV2019 [github] Shengyu Zhao, Yue Dong, Eric I-Chao Chang, Yan Xu

- Medical Registration

  • MICCAI2019

  • Image-and-Spatial Transformer Networks for Structure-Guided Image Registration,

  • Probabilistic Multilayer Regularization Network for Unsupervised 3D Brain Image Registration

  • A Deep Learning Approach to MR-less Spatial Normalization for Tau PET Images

  • TopAwaRe: Topology-Aware Registration

  • Multimodal Data Registration for Brain Structural Association Networks

  • Dual-Stream Pyramid Registration Network

  • A Cooperative Autoencoder for Population-Based Regularization of CNN Image Registration

  • Conditional Segmentation in Lieu of Image Registration

  • DeepAtlas: Joint Semi-supervised Learning of Image Registration and Segmentation

  • Linear Time Invariant Model Based Motion Correction (LiMo-MoCo) of Dynamic Radial Contrast Enhanced MRI

  • Incompressible Image Registration Using Divergence-Conforming B-Splines

  • Closing the Gap between Deep and Conventional Image Registration using Probabilistic Dense Displacement Networks [Codes]

- Point-cloud Registration

  • [2019ICCV] DeepVCP: An End-to-End Deep Neural Network for Point Cloud Registration
  • CMRNet: Camera to LiDAR-Map Registration, IEEE ITSC2019

Image Stitching

CNN Stitching

  • [2020] Attentive Deep Stitching and Quality Assessment for 360° Omnidirectional Images, JSTSP
  • Secondary
  • [2019] StitchNet: Image Stitching using Autoencoders and Deep Convolutional Neural Networks, thesis
  • [2018] Learning-based Natural Geometric Matching with Homography Prior, ElectronicsLetter
  • [2016] Image stitching with single-hidden layer feedforward neural networks, IJCNN

Spatially-Varying Warps

  • [2020TIP] Local-Adaptive Image Alignment Based on Triangular Facet Approximation
  • [2020JINS] Shape-optimizing mesh warping method for stereoscopic panorama stitching
  • [2019TVCG] Content-Preserving Image Stitching with Regular Boundary Constraints
  • [2019CGI] Stereoscopic Image Stitching with Rectangular Boundaries
  • [2019TMM] A Novel Projective-Consistent Plane based Image Stitching Method
  • [2018TMM] Parallax-Tolerant Image Stitching Based on Robust Elastic Warping [Code]
  • [2018TMM] Quasi-Homography Warps in Image Stitching
  • [2018ECCV] Image Stitching with Multiple Registrations [Homepage]
  • [2018ECCV] Object-Centered Image Stitching [Homepage]
  • [2018ECCVW] GreenWarps: A Two-Stage Warping Model for Stitching Images using Diffeomorphic Meshes and Green Coordinates
  • [2018Acce] An Accurate Multi-Row Panorama Generation Using Multi-Point Joint Stitching
  • [2018Acce] Generalized Content-Preserving Warps for Image Stitching
  • [2018Conf] Large Parallax Image Stitching Using an Edge-Preserving Diffeomorphic Warping Process
  • [2017] Robust Multi-homography Method for Image Stitching under Large Viewpoint Changes, conf
  • [2014TMM] Image Alignment by Piecewise Planar Region Matching

Seam-based Stitching

  • [2016ECCV] SEAGULL: Seam-guided Local Alignment for Parallax-tolerant Image Stitching [Note]

  • [2014CVPR] Parallax-tolerant Image Stitching [Proj/Data]

  • [2013EG] Seam-Driven Image Stitching

Dsitortion Preserving

  • [2019TIP] Single-Perspective Warps in Natural Image Stitching [Code] [Code-multi]

  • [2016ECCV] Natural Image Stitching with the Global Similarity Prior [Proj] [MacOS] [CCC] [VS]

  • [2005TOG] As-Rigid-As-Possible Shape Manipulation; [JoG2009] Implementing As-Rigid-As-Possible Shape Manipulation and Surface Flattening [Proj]

Warping/Projection

  • [2009CGF] A Shape-Preserving Approach to Image Resizing [Proj/Code]

  • [2005ICCV] Squaring the Circle in Panoramas

  • [2005TOG] As-rigid-as-possible shape manipulation [Proj]

Stitching/Panoramas

  • [2016TOG] Jump: Virtual reality video

  • [2016ACCV] Pano2Vid: Automatic Cinematography for Watching 360° Videos [Proj] [Code]

  • [2014ECCV] PanoContext: A Whole-room 3D Context Model for Panoramic Scene Understanding, oral [Proj/Code]

  • [2014ECCV] Photo Uncrop [Proj]

  • [2010ECCV] Real-Time Spherical Mosaicing Using Whole Image Alignment

  • [2010CVPR] Generating Sharp Panoramas from Motion-blurred Videos [Proj]

  • [2008MM] Discovering panoramas in web videos [Proj]

  • [2007EG] Scene Collages and Flexible Camera Arrays [Proj] [Sofeware]

  • [2007CVPR] Layered Depth Panoramas [Proj]

  • [2007Project] Automating Joiners [Proj] [Code] [PDF]

  • [2006TOG] Photographing long scenes with multi-viewpoint panoramas [Proj]

Stitching Application

  • [2016] Introducing facebook surround 360: An open, high-quality 3d-360 video capture system [Page]

  • [2012CVPR] Recognizing Scene Viewpoint using Panoramic Place Representation [Proj/Code]

Stereo Stitching

  • [2015CVPR] Casual Stereoscopic Panorama Stitching [Proj/Data]

Image Composition

Seamline

  • Coarse-to-fine Seam Estimation for Image Stitching, arXiv2018.5

  • [2019SIVP] Quality evaluation-based iterative seam estimation for image stitching [Code]

  • [2018SIVP] Perception-based seam cutting for image stitching [Code]

  • [2017IPOL] Efros and Freeman Image Quilting Algorithm for Texture Synthesis [2001Dynamic Seamline] [Code]

  • [2012TOG] Panorama weaving: fast and flexible seam processing [Proj]

Blending

- Gradient-based

  • [2019MM] GP-GAN: Towards Realistic High-Resolution Image Blending [Proj] [github] ⭐
  • [2012TOG] Image Melding: Combining Inconsistent Images using Patch-based Synthesis [Proj/Code]
  • [2010ECCV] Error-tolerant Image Compositing [Code]
  • [2007TOG] Efficient gradient-domain compositing using quadtrees [in PS CS3] [Proj]
  • [2004SIGG] Interactive Digital Photomontage [Proj&Code]
  • [2004ECCV] Seamless image stitching in the gradient domain
  • [2003SIGG] Poisson image editing

Dodging

  • [2019ISPRSJ] A Closed-Form Solution for Multi-View Color Correction with Gradient Preservation [Code]

    [2017ICCVW] Color Consistency Correction Based on Remapping Optimization for Image Stitching

  • [2018TIP] Dissecting and Reassembling Color Correction Algorithms for Image Stitching [Code&Supp]

  • [2016CVPR] Efficient and Robust Color Consistency for Community Photo Collections [Homepage] [Code]

  • [2011ICCV] Revisiting radiometric calibration for color computer vision

Stitching Assessment

  • [2019MM] Cross-Reference Stitching Quality Assessment for 360° Omnidirectional Images [Slides] [Dataset]

  • [2017ICCVW] A content-aware metric for stitched panoramic image quality assessment

  • Secondary

  • Quality Evaluation for Stitched Panoramic Videos, Thesis2018.11

Video Stitching

  • [2019BMVC] [CNNStitch] Video Stitching for Linear Camera Arrays [CNN-based] [Proj] ⭐

  • [2018TIP] Dynamic Video Stitching via Shakiness Removing [github]

  • [2017CGF] High-resolution 360 video foveated stitching for realtime VR

  • [2016TIP] Joint video stitching and stabilization from moving cameras [Proj] [Shuaicheng Liu]

  • [2016TOG] Rich360: Optimized spherical representation from structured panoramic camera arrays

  • [2016CGF] Seamless video stitching from hand-held camera inputs [Proj]

  • [2015CVPRW] Video Stitching with Spatial-Temporal Content-Preserving Warping

  • [2015EG] Panoramic Video from Unstructured Camera Arrays [Homepage]

  • [2015TOC] Efficient video stitching based on fast structure deformation

  • Secondary

  • [2018] Robust spatial–temporal bayesian view synthesis for video stitching with occlusion handling, MVA

Video Stabilization

  • StableNet: Semi-Online, Multi-Scale Deep Video Stabilization, arXiv2019.7

  • [2019SIGGAsia] Deep Iterative Frame Interpolation for Full-frame Video Stabilization

  • [2019TIP] Deep Online Video Stabilization With Multi-Grid Warping Transformation Learning [Code] [DebugNote] [Note] Deep Online Video Stabilization, arXiv2018

  • [2019CVPR] Robust Video Stabilization by Optimization in CNN Weight Space [Video]

  • [2018TOG] Joint Stabilization and Direction of 360◦ Videos

  • [2018CGF] Deep Video Stabilization Using Adversarial Networks [Note]

  • [2018] Deep Learning in Video Stabilization Homography Estimation, ConfNNA2018

  • [2015TIP] Video Stabilization Based on Feature Trajectory Augmentation and Selection and Robust Mesh Grid Warping

  • [2013CVPR] Plane-Based Content Preserving Warps for Video Stabilization

  • [2013TOG] Bundled camera Paths for Video Stabilization [Proj/Code] [As-similar-as-possible warping]

  • [2013TVCG] Spatially and Temporally Optimized Video Stabilization [Proj]

  • [2009TOG] Content-Preserving Warps for 3D Video Stabilization [Proj/Data]

Libs

  • SmartBlend

    It is an application for seamless image blending by Michael Norel. Smartblend allows stitching of many problematic shots (parallaxed, with moving objects or with differing exposure).

  • GCO3.0

    Matlab code for Max-flow/min-cut

Stitching Docs/Examples

Blogs

Examples

  • Panorama stitching Project (Python) [Code] ⭐

  • OpenPano: Automatic Panorama Stitching From Scratch [Code] ⭐⭐

  • Image-Alignment-and-Panoramas [Code]