/panorama-stitching

Panoramic image stitching using traditional and deep learning approach

Primary LanguageJupyter Notebook

License: MIT

Panorama-image stitching

The repository contains code to implement panaromic stitching using two approaches. First is the traditional method of image stitching using corner detection, Adaptive non-maximal suppression, feature descriptor, matching and RANSAC. The another is Deep Learning Approach where supervised and unsupervised approach is explored (HomographyNet).

Phase 1: Traditional methood

Sample dataset

Corners Detection

alt

Adaptive Non-Maximal Suppression

alt

Feature Descriptor

After we get the corner points, we need a descriptor to describe the feature for each point. To obtain that, a patch of size 40 × 40 centered at each corner point is used. This patch is then blurred and sub-sampled to a dimension of 8×8, which is then flattened to obtain a 64 × 1 vector.

Feature Matching

alt

RANSAC for outlier rejection and to estimate Robust Homography

alt

Blending Images

alt alt

Instructions to run the code:

  • Change the location to the root directory
  • Run the following command:
python3 Wrapper.py --BasePath ./Phase1/ --ImagesFolder Data/Train/Set3 --SaveFolderName Code/Results/Set3

Parameters

  • BasePath : Location for Phase 1. Eg. /home/abhijit/courses/CMSC733/sakshi_p1/Phase1/
  • ImagesFolder: Location for image folder relative to the BasePath. Eg Data/Test/TestSet2
  • SaveFolderName: Location where you want to save the results relativ BasePath. Eg. Code/Results/TestSet2
  • ShowImages: If you want to view the step outputs. Set as False by default
  • GoSequentially: Go sequentally while stitching or use half split method. Set as false by default.

Phase 2: Deep learning approach

We implemented two deep learning approaches to estimate the homography between two images. The deep model effectively combines corner detection, ANMS, feature extraction, feature matching, RANSAC and estimate homography all into one. This not only makes the approach faster but also makes it robust if the network is generalizable.

Data generation

  • Copy Train, Val and Phase2 (Test data folder was named so in zip file) Folders, to Phase2/Data/
  • cd Phase2/Code
  • To generate required patches and labels for training the models, run,
    python3 DataGenerator.py

Supervised

alt

Training

python3 Train.py --BasePath ../Data/Train_synthetic --CheckPointPath ../Checkpoints/supervised/ --ModelType sup --NumEpochs 100 --DivTrain 1 --MiniBatchSize 64 --LoadCheckPoint 0 --LogsPath ./Logs/supervised/

Testing

python3 Test.py --ModelPath ../Checkpoints/supervised/supervisedModel.h5 --BasePath ../Data/Test_synthetic --SavePath ./Results/ --ModelType sup

Unsupervised

alt

Training

python3 Train.py --BasePath ../Data/Train_synthetic --CheckPointPath ../Checkpoints/unsupervised/ --ModelType Unsup --NumEpochs 100 --DivTrain 1 --MiniBatchSize 64 --LoadCheckPoint 0 --LogsPath ./Logs/unsupervised/

Testing

python3 Test.py --ModelPath ../Checkpoints/unsupervised/0model.ckpt --BasePath ../Data/Test_synthetic --CheckPointPath ../Checkpoints/unsupervised/ --SavePath ./Results/ --ModelType Unsup

Results

alt