Final Project

Collision warning system for self-driving vehicle.

Project summary:

In this project we propose to construct a collision warning system for driverless vehicle to detect dynamic objects in the drivable region. Through the input video from driving recorder, the system can estimate the drivable space in 3D, provide category of obstacles present in lane, and eventually output “Go” and “Stop” command. To implement these function, we need to train convolutional neural networks for object detection and semantic segmentation.

Detail:

Focus on :

Novality of the network, usc bagnet to substitute the traditional cnn.

Reference:

Paper

Useful Website:

-1 Coursera courses:visual-perception-self-driving-cars
-2 KITTI数据集简介与使用
-3 KITTI数据使用序列——3D Object检测数据集使用

Machine:

-1 Mac osx: Program
-2 AWS p3.2xlarge GPU: Training Network
-3 Final Data: VOC Form KITTI
-4 GITHUB Fork : https://github.com/Weizhongjin/faster-rcnn.pytorch

Instruction of Useing this Model

This Faster-RCNN's Citation
@article{jjfaster2rcnn, Author = {Jianwei Yang and Jiasen Lu and Dhruv Batra and Devi Parikh}, Title = {A Faster Pytorch Implementation of Faster R-CNN}, Journal = {https://github.com/jwyang/faster-rcnn.pytorch}, Year = {2017} }

3D Object Detection

Our project has two part:

Point RCNN: we use code from : Point RCNN, and we just follow the instruction on its README.md
Faster-3D-RCNN:

we use the existed code of Faster-RCNN

For this part, the instruction will be :
1. git clone https://github.com/Weizhongjin/faster-rcnn.pytorch.git
// this is my fork of this faster-r-cnn and I add 2d-3d transform part within it.

or you can just use the file I upload.
1. ./ jwz.sh // Becuase they are lot of prerequisites and bugs, so I have writed all command line that's needed.
2. ```
CUDA_VISIBLE_DEVICES=0 python trainval_net.py --dataset pascal_voc --net vgg16 --bs 12 --nw 1  --cuda
// this is command line if you want to train a faster-rcnn
```
//more details look at https://github.com/jwyang/faster-rcnn.pytorch
```
4. If you want to use pretrained mode, you should downloads model to 'vgg16/pascal_voc/'

5. Then 

```mkdir transfermodel```

And downloads 2D-3D model to it.



6. Set test data

```shell
cd ..
mkdir images
##downloads imageset you want to test and unzip it.
```
1. run for result:
```
python 3d.py
```

Some Words

The novelty part are in 3d.py and 3dbox.py, we combine two networks together in 3d.py, 3dbox.py are the part we use opencv to draw 3d bounding box.
The lib has all files that relative to faster-r-cnn, and 2D-3D transform model is in 2dto3d file
In 2dto3d file we reference the 3DOD:
1. Transient.py has the framework of 2D-3D transform model(novelty)
This code will be really difficult to run successfully. Here will be several reasons:
1. The faster-rcnn using pytorch1.0 and all parts of this network using CUDA to accelerate . So if you use pytorch4.0 and has no GPU ,it cannot work successfully.
2. The image processing part use opencv-python, but this package will have some bugs when process different size images.
  
  It will show Segmentation fault
  
  For small images you can just use python3.6+pytorch1.0+cuda10.0, but for some larger image, the opencv-python may have some bugs s0 that you can only use python2.7_pytorch1.0+cuda9.0 to run the 3d.py. What's more you should also updata the gcc in your computer, more detail plz see jwyang/faster-rcnn.pytorch#409
Our work are mostly based on existed code, but we actually paid a lot of efforts. I spend a lot of time to fix bugs of faster-rcnn(the code I import are relatively new, the code contributor just posted this pytorch1.0 version recently, so it has a lot of bugs.) I also made some contribution to this Faster-RCNN GitHub.
The 2D-3D transform model are our own idea but 3DOD also finished this part with good performance, so I use part of his original code.
The work product is demo.mov
The I will continue on this project on summer, thank all your instructors and TAs.
If you have some problem or interest on this project, plz contact me: weizhonj@usc.edu

Citation:

@article{jjfaster2rcnn,
    Author = {Jianwei Yang and Jiasen Lu and Dhruv Batra and Devi Parikh},
    Title = {A Faster Pytorch Implementation of Faster R-CNN},
    Journal = {https://github.com/jwyang/faster-rcnn.pytorch},
    Year = {2017}
}

@inproceedings{renNIPS15fasterrcnn,
    Author = {Shaoqing Ren and Kaiming He and Ross Girshick and Jian Sun},
    Title = {Faster {R-CNN}: Towards Real-Time Object Detection
             with Region Proposal Networks},
    Booktitle = {Advances in Neural Information Processing Systems ({NIPS})},
    Year = {2015}
}

https://github.com/fregu856/3DOD_thesis

Weizhongjin/Driver-less-Project