This project was completed as part of Udacity Nanodegree Program. LIDAR and Camera detections from the Waymo Open Dataset (highway sequence) are fused for Tracking Objects in the Driving scene using an Extended Kalman filter. A Track Management and Data Association module has been implemented for handling multiple objects in the scene. It takes care of Track Initialisation and maintains a Track State (confirmed, tentitive, initialised)and Track score. False positive objects with low score or those that exit the driving scene are deleted.
The following diagram contains an outline of the data flow and of the individual steps that make up the algorithm.
All dependencies required for the project have been listed in the file requirements.txt.
The Waymo Open Dataset Reader is a very convenient toolbox that allows you to access sequences from the Waymo Open Dataset.The installation instructions can be found in tools/waymo_reader/README.md.
This project makes use of different sequences for Testing Algorithm: These are:
- Sequence 1 :
training_segment-1005081002024129653_5313_150_5333_150_with_camera_labels.tfrecord - Sequence 2 :
training_segment-10072231702153043603_5725_000_5745_000_with_camera_labels.tfrecord - Sequence 3 :
training_segment-10963653239323173269_1924_000_1944_000_with_camera_labels.tfrecord
To download these files, you will have to register with Waymo Open Dataset first: Open Dataset – Waymo
Once you have done so, please click here to access the Google Cloud Container that holds all the sequences. Once you have been cleared for access by Waymo (which might take up to 48 hours), you can download the individual sequences.
The sequences listed above can be found in the folder "training". Please download them and put the tfrecord-files into the dataset folder of this project.
📦project
┣ 📂dataset --> should contain the downloaded Waymo Open Dataset sequences
┃
┣ 📂misc
┃ ┣ evaluation.py --> plot functions for tracking visualization and RMSE calculation
┃ ┣ helpers.py --> misc. helper functions, e.g. for loading / saving binary files
┗ params.py --> parameter file for the tracking
┣ 📂results --> binary files with pre-computed Lidar Detections
┃
┣ 📂student
┃ ┣ association.py --> data association logic for assigning measurements to tracks
┃ ┣ filter.py --> extended Kalman filter implementation
┃ ┣ measurements.py --> sensor model and measurement processing for camera and lidar
┃ ┗ trackmanagement.py --> track and track management classes
┃
┣ 📂tools --> external tools
┃ ┗ 📂waymo_reader --> functions for loading Waymo data sequences
┃
┣ loop_over_dataset.py --> Main script for Object Tracking
This project uses pre-computed lidar detections of Waymo Sequences so everyone can use same input data for implementing Tracking Algorithm download the pre-computed lidar detections (~1 GB), unzip them and put them in the folder results.
For Camera measurements, noise is added the Waymo Image Labels and resulting noisy data is used as input for Sensor Fusion and Tracking.
Please execute loop_over_dataset.py to run the project
The Algorithm sucessfully deletes false positive Lidar or Camera detections which have a low Track Score and High estimation Covariance.
New Tracks are sucessfully initialised for handling new Cars that enter the driving scene
The state estimations were compared to Ground truth data and accumulated trackwise RMSE values for the entire sequence have been plotted below.
Parts of this project are based on the following repositories: