For Full detailed report please refer [Skylark GCP Report.pdf]
Manual detection of Ground Control Points is a cumbersome task and thus, a Computer Vision and Deep Learning model can help simplify this task by identifying the points and then plotting their co-ordinates.
Problem with using object detection: The GCP points are very small and thus, normal object detection algorithms could not be applied. There is scope of applying RetinaNet model, but that has led to bad accuracy, and great computation since it uses detection and here the method can be optimised using classification instead. (Classifying whether it is a GCP(1) or not(0)).
Pre-process Flowchart refer: pre-process flow chart.pdf
- Detection of contours in image: The contours can be used to detect white "L" shape GCP and crop that from the image for training images.
- Step 1 : Thresholding the RGB image and mask it with (220,255): the white mask. The value range 220-255 can be altered for images (this suited best for dataset provided).
- Step 2: Thresholding is followed by morphology. Used opencv function
closing = cv2.morphologyEx(img,cv2.MORPH_CLOSE,kernel) #Dilation: Useful for closing small holes in image in thresh image
opening = cv2.morphologyEx(closing, cv2.MORPH_OPEN,kernel) #Erosion: Useful in noise reduction
- Step 3: Find contours in the threshold image received from above.
- Step 4: Eliminate extra contours on basis of area and concativity. Since "L" is a concave structure. Rejected contours also are taken into consideration for developing train module for 0 (not L).
- Step 5: The required contours are then matched with given GCP for closest node. Here the required bounding box is attatched.
- Step 6: Crop contour of the required bounding box and prepare training module for 1 (not L).
Note: The coordinates are rounded to nearest integer to get output.
Notebook
The learning model is a Sequential model with 5 convolutional layers build in keras. To get a single max probability output, softmax is used. The filters kernel size in the convolutional layer are on trial and test basis. The training data has 286 positive samples and 213 negative samples contained in repository 'data/train'. The test samples has 8 samples each of negative and positive L, contained in repository 'data/test'. The model summary is
- The images tested from CV-Assignment-Dataset are stored with marked detection in plot directory.
- The model is saved as model.hdf5 file.
- Fill the original directory in orig_dir variable.
- Fill the destination of plot directory in dest_dir.
- The pre-processing is same as training.
- The segregated contours are then passed to model for prediction of required shape.
- The minAreaRect function gives box co-ordinates of the prediction in the threshold image.
- Detect Harris Corners in the predicted contour and eliminate all corners that lie beyond the boundaries of the above calculated box.
- The Harris corner points are dilated and then passed to a cv2.cornerSubPix() function to get corners with Sub Pixel Accuracy (Source: Link)
- Calculate Center of Mass of the predicted contour, since the required point is always close to COM.
- Calculate the corner (detected from Harris) closest to COM, that's the required co-ordinates.
- Encapsulate the image name and coordinates to output.csv file.