Finding Lane Lines on the Road
During the beginning of the self-driving car Engineer Nanodegree, learned a series of image processing techniques. I had the opportunity to apply this knowledge in a very interesting and hands-on project, to find traffic lanes on images and in a video stream. For that, I have programmed an image processing pipeline to treat the images and video streams to detect the lanes.
Image processing pipeline
My pipeline consisted of 5 steps. We start with this image, and on each step, I will show the result of the step on the image.
1. Converting to grayscale
Converting to grayscale allows us to handle the image more easily with the techniques that will follow.
I've used cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
, the simplest way to convert to grayscale. In the future,
I might re-visit this part of the code and use more sophisticated transformations or maybe a method based
in clustering and thresholding, like Otsu's method.
2. Gaussian filtering
In this step, I applied a gaussian filter, which smoothes the image and creates a blur.
This is necessary to reduce image noise and remove unwanted results.
I've used cv2.GaussianBlur(img, (kernel_size, kernel_size), 0)
,
with kernel_size=11
. A larger kernel would apply a stronger smoothing.
3. Canny detection
The Canny edge detector is a method to find edges in an image.
It is based on the gradient of pixel intensities. I've used cv2.Canny(img, low_threshold, high_threshold)
.
low_threshold
and high_threshold
are parameters set to filter the gradient values, so only values between
low_threshold
and high_threshold
are kept. This part was critical, because depending on the road conditions,
the Canny algorithm would ignore the lanes, so a proper calibration is essential.
4. Mask
On this project, we assume that the camera is mounted in a fixed and know position of the car. Given that, we can expect that the lanes will be inside a polygon:
Thus, we can filter the Canny transform result to have the following result. We can see that much of the ambient, like trees and the horizon, which would trick our algorithm, is removed from the image.
5. Hough transform and final result
The Hough transform is used for feature extraction in computer vision to find classes of shapes in a given picture. These shapes can be circles, polygons, but in this case, I used Hough transform to detect lines from the masked Canny result image to find the actual driving lanes. After using the Hough transform, we will also filter the lines, to detect the right and left lane lines, using an average filter and a line interpolation. The final result is:
Try it yourself
Please follow instructions here to set up your conda environment.
Then, on your terminal use conda activate carnd-term1
to activate the conda environment, then run jupyter notebook
to see the results in jupyter or just run the P1.py file using python3 P1.py
. The images will appear on test_images_output
folder and the videos in test_videos_output.
Shortcomings of the pipeline
This pipeline was specifically designed and calibrated for this specific set of example images and videos. Thus, in different lighting conditions, or even different pavement color, this algorithm can behave unexpectedly. I personally don't expect that it will work at all in low light conditions like in the dark. Moreover, objects on the road or even dirt can potentially confuse the pipeline.
Possible improvements
One immediate improvement I can propose is to find a way to automatically calibrate the Canny transform based on the current lighting of the image. I could see that there are some pages about auto-Canny, like here but I expect that during this Self-Driving Car Engineer Nanodegree we will learn more exciting and effective techniques to treat that. Another possible improvement would be to have a configuration file instead of the Configuration class I'm using. I haven't done that because I did not want to include other dependencies than the ones from the course example, but in a real implementation I would definitely use configuration files instead of changing the code for that.