In this project (#4 in Udacity's SDCND) I develop a pipeline to detect the road lane and lane features from images taken within a driving vehicle. The pipeline processes single images but can also be applied to frames of a video.
If you want to toy with the code, it's all in pipeline.ipynb. The code is well-documented and there's lots of supporting text to walk you through the notebook. [Make sure you get all the dependencies installed. My repo of Udacity's starter might help.]
A Medium write-up of this project can be found here.
The lane-detection pipeline consists of the following steps:
- Pre-Step: Calibrate the camera
- Undistort the image
- Threshold the image using gradients and colors
- Apply a perspective transform to view the image from top-down
- Identify the lane lines in warped image
- Draw onto the original image
The pre-step, calibrating the camera, is performed only once (assuming all the images are taken from the same camera). All the other steps are performed on each image.
"Calibrating the camera" really means accounting for the distortion in an image introduced by the camera's lens. This is done using multiple images of checkerboard patters, which should have straight lines. Examining how the checkerboard patterns are distorted allows us to precisely identify how the camera lens is distorting images - which means we can undistort them.
With the camera calibrated, we simply need to apply the knowledge gained to undistort images. This is important because it will restore the straightness of lines, helping to identify lane lines later in the pipeline. The difference between the distorted and undistorted images is clear. The bent lines are now straight.
| Distorted | Undistorted |
|---|---|
|
 |
Thresholding is a method of narrowing down the pixels we are interested in. This can be done using a combination of gradient and color filters. Here's what a thresholded image looks like.
| Original | Thresholded |
|---|---|
 |
 |
While undistorting and thresholding help isolate the important information, we can further isolate that information by looking only at the portion of the image we care about - the road. To focus in on the road portion of the image we shift our perspective to a top-down view of the road in front of the car.
| Thresholded - Normal Perspective | Thresholded - Top-Down Perspective |
|---|---|
|
 |
From the top-down perspective it's much easier to identify the lane lines. Lane curvature is also much easier to calculate. Here is an image showing the lane lines identified by a sliding window search.
Finally, we take all this information we gathered and draw the results back onto the original image. The calculated right/left lane curvature and center-lane offset are shown in the top-left of the image.
At the end of pipeline.ipynb is some simple code that processes videos using this lane-detection pipeline. My final video output for three different videos can be found on YouTube here: