Capstone

This is the project repo for the final project of the Udacity Self-Driving Car Nanodegree: Programming a Real Self-Driving Car. For more information about the project, see the project introduction here.

Team Members

Yuan (Jack) Li (yuanli12139@gmail.com)

Peng Zhang (pengzhang40@gmail.com)

ROS System Architecture

Traffic Light Detection

We use YOLO v2 tiny detector as our traffic light detector (https://pjreddie.com/darknet/yolov2/) for its real-time efficiency. We download the cfg file and weights pretrained on COCO dataset as our baseline.

Dataset

We combine the Udacity Autonomous Driving Dataset 2 (https://github.com/udacity/self-driving-car/tree/master/annotations) and the traffic light dataset collected by https://github.com/coldKnight/TrafficLight_Detection-TensorFlowAPI, and then use sort_data.py to convert the annotations into trainable .xml files.

Training

Follow the instructions of darkflow (https://github.com/thtrieu/darkflow) and modify labels and config file accordingly.

./flow --model cfg/yolov2-tiny_ft.cfg --train --load bin/yolov2-tiny.weights --dataset "/mnt/data/datasets/CarND-Capstone/Data" --annotation "/mnt/data/datasets/CarND-Capstone/Annotations" --gpu 1.0 --lr 5e-4 --epoch 10 --save 1000 --trainer adam

We also trained YOLO v2 full version but it underperforms the Tiny YOLO v2. This is potentially due to overfitting with the limited training data.

Our checkpoint files can be found at

https://drive.google.com/open?id=1Z5EbbLe9BPH5O4FQWEWn41-uw8KBkoXC

https://drive.google.com/open?id=1JniOI-5w8npWoqUHZaGepMcaIGj5mbmj

Installation

Please use one of the two installation options, either native or docker installation.

Native Installation

Be sure that your workstation is running Ubuntu 16.04 Xenial Xerus or Ubuntu 14.04 Trusty Tahir. Ubuntu downloads can be found here.
If using a Virtual Machine to install Ubuntu, use the following configuration as minimum:
- 2 CPU
- 2 GB system memory
- 25 GB of free hard drive space
The Udacity provided virtual machine has ROS and Dataspeed DBW already installed, so you can skip the next two steps if you are using this.
Follow these instructions to install ROS
- ROS Kinetic if you have Ubuntu 16.04.
- ROS Indigo if you have Ubuntu 14.04.
Dataspeed DBW
- Use this option to install the SDK on a workstation that already has ROS installed: One Line SDK Install (binary)
Download the Udacity Simulator.

Docker Installation

Install Docker

Build the docker container

docker build . -t capstone

Run the docker file

docker run -p 4567:4567 -v $PWD:/capstone -v /tmp/log:/root/.ros/ --rm -it capstone

Port Forwarding

To set up port forwarding, please refer to the instructions from term 2

Usage

Clone the project repository

git clone https://github.com/udacity/CarND-Capstone.git

Install python dependencies

cd CarND-Capstone
pip install -r requirements.txt

Make and run styx

cd ros
catkin_make
source devel/setup.sh
roslaunch launch/styx.launch

Run the simulator

Real World Testing

Download training bag that was recorded on the Udacity self-driving car.
Unzip the file

unzip traffic_light_bag_file.zip

Play the bag file

rosbag play -l traffic_light_bag_file/traffic_light_training.bag

Launch your project in site mode

cd CarND-Capstone/ros
roslaunch launch/site.launch

Confirm that traffic light detection works on real life images

yuanli12139/CarND-Capstone