This project is developed for tracking multiple objects in 3D scene. The visualization code is from here. This is a re-produced and simplified version of paper. We have made some changes to support more detectors and adapt the HOTA metric. We also provided a simple guide to submit results to the KITTI test set.
- Fast: currently, the codes can achieve 700 FPS using only CPU (not include detection and data op), can perform tracking on all kitti val sequence in several seconds.
- Support online, near online and global implementation. The overall framework of design is shown below:
Car/Pedestrian tracking results on the Kitti tracking val seq [1,6,8,10,12,13,14,15,16,18,19] using second-iou, point-rcnn and pv-rcnn detections. We also followed the HOTA metric, and tuned the parameters by first considering the HOTA performance.
detector | online | near online (latency=2s) | global |
---|---|---|---|
point-rcnn | 76.653 | 77.576 | 78.853 |
second-iou | 77.29/48.011 | 78.17/50.362 | 78.799/51.20 |
pv-rcnn | 78.289/49.662 | 79.48/50.824 | 80.075/51.753 |
Online: CA-based KF + greedy matching. Near online: online + rescoring tracks in a temporal window. Global: online + rescoring tracks globally.
- 2022/9/23 Update CasTrack. The detections are obtained by CasA detector trained on KITTI 3D detection trainval set. Currently rank first on the KITTI tracking dataset π₯!
Detector | online | near online (latency=2s) | global | detections (training & testing set) |
---|---|---|---|---|
CasA | 82.59 | 83.24 | 83.34 | detections download link(40M) |
- Update VirConvTrack (2023/3/7), the detections are obtained by VirConv-T detector trained on KITTI 3D detection trainval set.
Detector | online | near online (latency=2s) | global | detections (training & testing set) |
---|---|---|---|---|
VirConv | 85.40 | 85.98 | 86.17 | detections download link(8M) |
Vehicle/Pedestrian/Cyclist online tracking results, evaluated by MOTA/L2 metrics of Waymo benchmark.
- 2022/9/23 Update CasTrack-waymo, currently ranks third on Waymo tracking benchmark π₯! You can download the CasA detections from here (Please register with waymo before downloading, as the Waymo open dataset is under strict non-commercial license).
detector | set | Vehicle_L2 | Pedestrian_L2 | Cyclist_L2 | ALL_NS_L2 |
---|---|---|---|---|---|
CasA | val set | 59.30 | 62.78 | 61.72 | 61.27 |
CasA | test set | 63.66 | 64.79 | 59.34 | 62.60 |
You can download the Kitti tracking pose data from here, and you can download the point-rcnn, second-iou and pv-rcnn detections from here. You can download the CasA detections(including training & testing set) from here.
To run this code, you should organize Kitti tracking dataset as below:
# Kitti Tracking Dataset
βββ kitti_tracking
βββ testing
| βββcalib
| | βββ0000.txt
| | βββ....txt
| | βββ0028.txt
| βββimage_02
| | βββ0000
| | βββ....
| | βββ0028
| βββpose
| | βββ0000
| | | βββpose.txt
| | βββ....
| | βββ0028
| | βββpose.txt
| βββlabel_02
| | βββ0000.txt
| | βββ....txt
| | βββ0028.txt
| βββvelodyne
| βββ0000
| βββ....
| βββ0028
βββ training # the structure is same as testing set
βββcalib
βββimage_02
βββpose
βββlabel_02
βββvelodyne
Detections
βββ point-rcnn
βββ training
| βββ0000
| | βββ000001.txt
| | βββ....txt
| | βββ000153.txt
| βββ...
| βββ0020
βββtesting
python3
numpy
opencv
yaml
- Please modify the dataset path and detections path in the yaml file to your own path.
- Then run
python3 kitti_3DMOT.py config/online/pvrcnn_mot.yaml
- The results are automatically saved to
evaluation/results/sha_key/data
, and evaluated by HOTA metrics.
The evaluation codes are copied from Kitti.