- A submission by Jai Lad
- Objective(s)
- Key File(s)
- Pipeline
- Extended pipeline
- Analyzing trend of fits for project video.
- Future work
The goals / steps of this project are the following:
- Compute the camera calibration matrix and distortion coefficients given a set of chessboard images.
- Apply a distortion correction to raw images.
- Use color transforms, gradients, etc., to create a thresholded binary image.
- Apply a perspective transform to rectify binary image ("birds-eye view").
- Detect lane pixels and fit to find the lane boundary.
- Determine the curvature of the lane and vehicle position with respect to center.
- Warp the detected lane boundaries back onto the original image.
- Output visual display of the lane boundaries and numerical estimation of lane curvature and vehicle position.
- output_images folder - Folder with various images as generated from the pipeline.
- writeup.md - Report writeup file
- project.ipynb - Jupyter notebook used to implement the pipeline.
- project.py - Python script version of the above notebook. ( useful for referencing specific file numbers )
- pipeline video - Video of the pipeline working on the project video.
- harder challenge video - Video of the pipeline working on the harder challenge video.
- challenge_images - A folder containing some challenging images extracted from the challenge videos.
- Code links, images and detailed description for the pipeline are available in: writeup.md
- Here, I used OpenCV's 'findChessboardCorners' method to detect object points and image points, followed by 'calibrateCamera' to determine the Camera calibration parameters.
- Here, I used the calibration parameters along with OpenCV's 'undistort' method to test distortion correction on an image.
- Here, I used a combination of Sobel-X gradient, and S-Channel thresholding to generate a thresholded binary image. I tweaked the thresholding parameter(s) until I was obtaining good results on a wide variety of images.
- Here, I focussed my attention to points of interest, which would be relevant from a lane detection perspective, and then rejected the other points.
- Here, I converted the region of interest image, into a bird's eye view image. This was done, so that the the left and right lane lines, which actually are parallel, appeared parallel as well. This helped in fitting polynomial curves to the detected lane lines.
- This was done to separate the lanes into left and right lanes.
- This was done to determine a good binomial fit for the detected lane lines.
- Here, I used the polynomial fitted above, to synthesize continuous left and right lane lines.
- Here, I filled the area in between the left, and the right lane line(s), so that this area was marked clearly.
- Here, I projected back the lane line(s) onto the original image, using an inverse perspective transform.
- Here, I performed radius of curvature and vehicle location calculations.
- Here, I used the techniques described above, to create an end-to-end pipeline which could be used with images and video, and which could also be used to generate visualizations of all the intermediate step(s).
- Code links and detailed description for the extended pipeline are available in : writeup.md
- Here, I kept a track of recent valid frame(s) and use a mean of these value(s) to determine if the current bottom most intersection point for the left lane and the right lane make(s) sense. If it does not make sense, then we use a mean of historical value(s). If it makes sense, then we pop the oldest entry, and update the history with this latest valid value.
- If we decide that the current detected polynomial make(s) sense, then we add it to the recent valid history, and then we generate a mean of the recent valid values. This allows the transitions to be smooth, and not 'jumpy'.
- In this section, I analyzed the various fit parameters to assess the trend of left fit coefficients and right fit coefficients as generated by the pipeline for the project video. This was used to tune the logic for validating and smoothing lane detections.
- Currently, I am using a hard-coded region of interest, however, for cases when we have significant curves, it is better to have an adaptive region of interest based on radius of curvature determination.
- Currently, I am using a hard-coded values for thresholded binary generation ( Sobel-X threshold, and S-Channel threshold ), however, for cases when we have a scene with less contrast, or a dark scene, it is better to have an adaptive threshold which adapts to the current scene, and which keeps a trend of the changing contrast every scene.
- Currently, I am using a fixed size of 5 valid past lanes for the left and right lane lines. This works well when the curvature(s) in the scene(s) is generally smooth. However, when the curves are very aggressive, we need to reduce the history so that we can adapt rapidly to the changing lane curvature(s).
- The lane detection problem could be converted to one of predicting lane line(s) given a particular scene, and then training a deep CNN on it. The input feature can be the image after region of interest selection, and the labelled output can be the 'fit line'. Infact, the pipeline for this project, can be used to quickly label a lot of images, which when combined with other tranformations like blurring, contrast changes, rotation(s), stretching etc, could be used to generate a sizable labelled data set. It would definitely be very interesting to see how this approach performs for this problem.
- Currently, I am keeping a history of past 'X' valid left and right lane points. However, all of these are weighted equally. A better approach, would be to combine this history with a 'decay' mechanism, so that more recent value(s) are given a higher weight, relative to older valid entries.
- In the harder challenge video, towards the end, we temporarily have no right lane line. The pipeline would have to be adapted for this scenario because we currently assume that we should be able to threshold in a manner so as to yield a decent lane fit.