3D Object Detection
Getting Started
Implemented and tested on Ubuntu 16.04 with Python 3.5.5 and Tensorflow 1.3.0.
- Clone this repository
git clone git@github.com:leofansq/3D-Detection.git
- Compile C++ files in wavedata
cd wavedata/tools/core/lib
cmake src
make
- Add to your PYTHONPATH
export PYTHONPATH=$PWD:$PWD/wavedata
- We use Protobufs to configure model and training parameters. Before the framework can be used, the protos must be compiled (from top level avod folder):
sh avod/protos/run_protoc.sh
Training
Dataset
To train on the Kitti Object Detection Dataset:
- Download the data and place it in your home folder at
~/Kitti/object
- Go here and download the
train.txt
,val.txt
andtrainval.txt
splits into~/Kitti/object
. Also download theplanes
folder into~/Kitti/object/training
The folder should look something like the following:
Kitti
object
testing
training
calib
image_2
label_2
planes
velodyne
train.txt
val.txt
Generate planes yourself
You can generate the plane files by yourself using RANSAC. The repository Tools _make _planes is to generate these files.
Mini-batch Generation
The training data needs to be pre-processed to generate mini-batches for the RPN. To configure the mini-batches, you can modify avod/configs/mb_preprocessing/rpn_[class].config
. You also need to select the class you want to train on. Inside the scripts/preprocessing/gen_mini_batches.py
select the classes to process. By default it processes the Car and People classes, where the flag process_[class]
is set to True. The People class includes both Pedestrian and Cyclists. You can also generate mini-batches for a single class such as Car only.
cd avod
python scripts/preprocessing/gen_mini_batches.py
Once this script is done, you should now have the following folders inside avod/data
:
data
label_clusters
mini_batches
Training Configuration
There are sample configuration files for training inside avod/configs
. You can train on the example configs, or modify an existing configuration. To train a new configuration, copy a config, e.g. example.config
, rename this file to a unique experiment name and make sure the file name matches the checkpoint_name: 'example'
entry inside your config.
Run Trainer
To start training, run the following:
python avod/experiments/run_training.py --pipeline_config=avod/configs/example.config --device='0' --data_split='train'
If the process was interrupted, training (or evaluation) will continue from the last saved checkpoint if it exists.
Evaluation
Run Evaluator
To start evaluation, run the following:
python avod/experiments/run_evaluation.py --pipeline_config=avod/configs/example.config --device='0' --data_split='val' --ckpt_start='60000'
The ckpt_start is the number of the model which you want to start evaluation. That means you can start your evaluation from any model, instead of starting from the begining.
View TensorBoard Summaries
To view the TensorBoard summaries:
cd avod/data/outputs/example
tensorboard --logdir logs
Results
The evaluator runs the KITTI native evaluation code on each checkpoint. Predictions are converted to KITTI format and the AP is calculated for every checkpoint.
To see the result txt:
cd avod/data/outputs/example/predictions/kitti_native_eval
sh all_eval.sh 0.1
0.1 is the score threshold. IoUs are set to (0.7, 0.5, 0.5) .
Inference
Run Inference
To run inference, run the following script:
python avod/experiments/run_inference.py --checkpoint_name='example' --data_split='val' --ckpt_indices=120 --device='0'
The ckpt_indices here indicates the indices of the checkpoint in the list. You can also just set this to -1 to evaluate the last checkpoint.
KITTI's Format
If you want to submit your result to KITTI sever, you need to transform the format to KITTI's format. Modify checkpoint_name and global_steps in save_kitti _predictions.py. Run the following script:
python save_kitti_predictions.py
Visualization
3D Detection Results
To visualize the 3D detection results, you can run show_predictions _2d.py. You need to configure the parameters to your specific experiments, and run the following script:
python show_predictions_2d.py
BEV Detection Results
To visualize the BEV detection results, you can run show_predictions _bev.py. You need to configure the parameters to your specific experiments, and run the following script:
python show_predictions_bev.py
AP Information
The plot_ap.py will plot the AP vs. step, and print the 5 highest performing checkpoints for each evaluation metric at the moderate difficulty. You need to configure the parameters to your specific experiments, and run the following script:
python plot_ap.py