Camera Calibration - Estimation of Camera Extrinsic and Intrinsic Parameters

Description:

Implementation of Zhang's Camera Calibration algorithm to estimate a camera's extrinsic parameter matrices "R", "t" and intrinsic parameter matrix "K". It uses a Calibration pattern of checkerboard to estimate these parameters

0. Generate dataset of images by using the Calibration Pattern in different views
1. Generate Hough lines and separate horizontal from vertical
2. From the hough lines, get rid of duplicate lines and then find intersection of horizontal and vertical lines to get corners corrdinates on image plane
3. Generate world coordinates of the corresponding corners in the image plane
4. Find Homography H between world plane Z=0 and image plane
5. Using this homography and the image of absolute conic, solve equations to estimate the image (omega) of the absolute conic.
6. Using the relationship between omega and K, obtain the intrinsic parameter matrix K. This is a estimate and will be refined further
7. Now, build an initial estimate of R and t matrices using K and H
8. Condition matrix R to be orthonormal
9. Convert the matrix R to a vector R_vec using Rodrigues formula
10. Build initial estimate of all the parameters, 5 for K, 3*no_of_images in dataset for R and 3*no_of_images_in_dataset for t
11. If incorporating radial distortion, the also include parameters k1 and k2
12. Optimize the geometric error between the actual corners and projected coordinates of world coordinates in the image plane using Levenberg - Marquardt algorithm
13. Rebuild the matrices R, t for each image from the 1D vector obtained after optimization. Note that the R_vect should be converted to R matrix using Rodrigues formula
14. Rebuild estimated K matrix from 1D vector after optimization

Original Paper: Zhang_technical_report.pdf

Refer here for the mathematical formulation and technical explanation

Dependencies

OpenCV
NumPy
SciPy (for Levenberg-Marquardt implementation)

Scripts

calibrate_camera.py: MAIN file to run. Pass the correct parent folder and image list in the script. Result folder with all intermediate images are generated in /<parent_folder>/results_<fixed_img_no>

python calibrate_camera.py

Supporting scripts

compute_corners.py: Script to get coordinates of corners by intersection of lines obtained using Hough transform
estimate_homography.py: Helper functions which help in bilinear interpolation and projecting images to a canvas using Homography matrix

Results

Input Images

Pic_28.jpg

Pic_40.jpg

Hough Lines and Processing

Corners and refinement

Corners before in **Magenta** (found by intersection of hough lines) and after refinement in **Green**. These corners would be used as ground truth image coordinates for the projection of world coordinates

World coordinates

The world coordinates are taken as [0, 0, 0, 0] for the point marked "0" in the above image and increased by 25 and 25 in x, y directions for subsequent labelled points in the above image. That is, "0": [0, 0, 0, 0], "1": [25, 0, 0, 0], "8": [0, 25, 0, 0] and so on.

Camera Intrinsic and Extrinsic Parameters

Camera Intrinsic Parameters (K) and Extrinsic Parameters ([R|t]) before and after Levenberg Marquardt optimization - Without Radial Distortion incorportated

Camera Intrinsic Parameters (K) and Extrinsic Parameters ([R|t]) before and after Levenberg Marquardt optimization - WITH Radial Distortion incorportated

Visual Results of Camera Parameters

No Radial Distortion included

Projecting the world coordinates of Pic_28.jpg using the calculated camera matrix. The left image is before the LM optimization for refinement of parameters. The right image is after the refinement of parameters. Blue are the estimated projection of world coordinates in image plane using camera matrix. Green are the ground truth image plane projections of world coordinates.

Projecting image coordinate estimation of world coordinates using Pic_40.jpg camera matrix onto Pic_28.jpg

aartighatkesar/Camera_Calibration