Super Fast and Accurate 3D Object Detection based on 3D LiDAR Point Clouds

Features

Super fast and accurate 3D object detection based on LiDAR
Fast training, fast inference
An Anchor-free approach
No Non-Max-Suppression
Support distributed data parallel training
Release pre-trained models

The technical details are described here

Update 2020.09.06: Add ROS source code. The great work has been done by @AhmedARadwan. The implementation is here

Demonstration (on a single GTX 1080Ti)

Youtube link

2. Getting Started

2.1. Requirement

The instructions for setting up a virtual environment is here.

git clone https://github.com/maudzung/SFA3D.git SFA3D
cd SFA3D/
pip install -r requirements.txt

2.2. Data Preparation

Download the 3D KITTI detection dataset from here.

The downloaded data includes:

Velodyne point clouds (29 GB)
Training labels of object data set (5 MB)
Camera calibration matrices of object data set (16 MB)
Left color images of object data set (12 GB) (For visualization purpose only)

Please make sure that you construct the source code & dataset directories structure as below.

2.3. How to run

2.3.1. Visualize the dataset

To visualize 3D point clouds with 3D boxes, let's execute:

cd sfa/data_process/
python kitti_dataset.py

2.3.2. Inference

The pre-trained model was pushed to this repo.

python test.py --gpu_idx 0 --peak_thresh 0.2

2.3.3. Making demonstration

python demo_2_sides.py --gpu_idx 0 --peak_thresh 0.2

The data for the demonstration will be automatically downloaded by executing the above command.

2.3.4. Training

2.3.4.1. Single machine, single gpu

python train.py --gpu_idx 0

2.3.4.2. Distributed Data Parallel Training

Single machine (node), multiple GPUs

python train.py --multiprocessing-distributed --world-size 1 --rank 0 --batch_size 64 --num_workers 8

Two machines (two nodes), multiple GPUs

First machine

python train.py --dist-url 'tcp://IP_OF_NODE1:FREEPORT' --multiprocessing-distributed --world-size 2 --rank 0 --batch_size 64 --num_workers 8

Second machine

python train.py --dist-url 'tcp://IP_OF_NODE2:FREEPORT' --multiprocessing-distributed --world-size 2 --rank 1 --batch_size 64 --num_workers 8

2.3.5 Evaluation

Do Evaluation on kitti val dataset

python  train.py --evaluate  --gpu_idx 0 --pretrained_path=../checkpoints/fpn_resnet_18/fpn_resnet_18_epoch_300.pth

Evaluation result

Car AP(Average Precision)@0.70, 0.70, 0.70:
bbox AP:96.57, 89.17, 89.41
bev  AP:97.52, 89.62, 89.78
3d   AP:88.09, 87.86, 88.09
aos  AP:60.28, 55.63, 55.04
Car AP(Average Precision)@0.70, 0.50, 0.50:
bbox AP:96.57, 89.17, 89.41
bev  AP:98.03, 89.94, 90.09
3d   AP:98.01, 89.94, 90.09
aos  AP:60.28, 55.63, 55.04
Pedestrian AP(Average Precision)@0.50, 0.50, 0.50:
bbox AP:65.38, 64.95, 63.96
bev  AP:68.14, 69.36, 65.41
3d   AP:66.44, 62.36, 62.77
aos  AP:32.13, 30.95, 30.21
Pedestrian AP(Average Precision)@0.50, 0.25, 0.25:
bbox AP:65.38, 64.95, 63.96
bev  AP:88.43, 88.61, 88.59
3d   AP:88.29, 88.45, 88.48
aos  AP:32.13, 30.95, 30.21
Cyclist AP(Average Precision)@0.50, 0.50, 0.50:
bbox AP:89.62, 87.64, 87.70
bev  AP:82.11, 75.41, 75.64
3d   AP:80.09, 74.31, 74.45
aos  AP:54.37, 52.01, 51.44
Cyclist AP(Average Precision)@0.50, 0.25, 0.25:
bbox AP:89.62, 87.64, 87.70
bev  AP:96.04, 88.67, 88.78
3d   AP:96.04, 88.67, 88.78
aos  AP:54.37, 52.01, 51.44

The original evaluation code is from here.

Tensorboard

To track the training progress, go to the logs/ folder and

cd logs/<saved_fn>/tensorboard/
tensorboard --logdir=./

Then go to http://localhost:6006/

Contact

If you think this work is useful, please give me a star!
If you find any errors or have any suggestions, please contact me (Email: nguyenmaudung93.kstn@gmail.com).
Thank you!

Citation

@misc{Super-Fast-Accurate-3D-Object-Detection-PyTorch,
  author =       {Nguyen Mau Dung},
  title =        {{Super-Fast-Accurate-3D-Object-Detection-PyTorch}},
  howpublished = {\url{https://github.com/maudzung/Super-Fast-Accurate-3D-Object-Detection}},
  year =         {2020}
}

References

[1] CenterNet: Objects as Points paper, PyTorch Implementation
[2] RTM3D: PyTorch Implementation
[3] Libra_R-CNN: PyTorch Implementation

The YOLO-based models with the same BEV maps input:
[4] Complex-YOLO: v4, v3, v2

3D LiDAR Point pre-processing:
[5] VoxelNet: PyTorch Implementation

Folder structure

${ROOT}
└── checkpoints/
    ├── fpn_resnet_18/    
        ├── fpn_resnet_18_epoch_300.pth
└── dataset/    
    └── kitti/
        ├──ImageSets/
        │   ├── test.txt
        │   ├── train.txt
        │   └── val.txt
        ├── training/
        │   ├── image_2/ (left color camera)
        │   ├── calib/
        │   ├── label_2/
        │   └── velodyne/
        └── testing/  
        │   ├── image_2/ (left color camera)
        │   ├── calib/
        │   └── velodyne/
        └── classes_names.txt
└── sfa/
    ├── config/
    │   ├── train_config.py
    │   └── kitti_config.py
    ├── data_process/
    │   ├── kitti_dataloader.py
    │   ├── kitti_dataset.py
    │   └── kitti_data_utils.py
    ├── models/
    │   ├── fpn_resnet.py
    │   ├── resnet.py
    │   └── model_utils.py
    └── utils/
    │   ├── demo_utils.py
    │   ├── evaluation_utils.py
    │   ├── logger.py
    │   ├── misc.py
    │   ├── torch_utils.py
    │   ├── train_utils.py
    │   └── visualization_utils.py
    ├── demo_2_sides.py
    ├── demo_front.py
    ├── test.py
    └── train.py
├── README.md 
└── requirements.txt

CarkusL/SFA3D