Using image warping and homographies, an image mosaic is created.
To run this appliction. You need to install python, cv2 and numpy
- Run main.py using the command
python main.py
- Two images will show up. Use the mouse to define correspondences. Click on any pixel in the first image then click on the corresponding pixel on the second image. You have to define at least 4 correspondences (8 clicks). The more pixels you define, the more accurate result you will get
- Press any key and wait until the output image appears (this may take a while).
- Press any key to close the output image.
To use other images:
- copy the images you want to the current directory.
- Change main.py lines 6 and 7 to the new image names
Using cv2 MouseCallback, all mouse clicks are stored in two lists: imgClicks1
and imgClicks2
. These lists are used to compute the homography matrix. This step is done in main.py
1.Matrix A is computed by stacking together constraints from multiple correspondences as follows:
A =
(Image credits: cmu slides http://www.cs.cmu.edu/~16385/)
where xi and yi stand for the coordinates of mouse clicks in the right images, xi` and yi` stand for the corresponding coordinates of mouse clicks in the second image.
- To compute the h vector, we computed the SVD of A.
A was decomposed into three matrices: U, S and V
- The vector corresponding to the smallest eigenvalue was stored and reshaped into a 3*3 matrix forming H, the homography matrix.
There are two ways to warp the right images: warping and inverse warping.
We know that the computed H can map any homogeneous coordinates in the left image to their correspondings in the right one using the formula:
p` = H.p
But using the forward warping, there will be holes in the warped image. Instead of the forward warping, we used inverse warping, in which the formula is:
p = inv(H). p`
Rather than using the coordinates of the original image and storing their values in the computed p`, we go over each pixel in the warped image (p`), compute their original coordinates in the original image and store the value in p`.
The final step is concatenating both the left image and warped right image in one image. The output is the image mosaic.