This repository contains the corresponding source code for the paper "Track to Reconstruct and Reconstruct to Track" arXiv PrePrint.
This paper has been accepted as both a conference paper at ICRA 2020, as well as being accepted as a journal paper in Robotics and Automation Letters (RA-L)!!!
The code was tested on:
- CUDA 9, cuDNN 7
- Tensorflow 1.13
- Python 3.6
Note: For some external code that is required to run in the precompute script, you need different requirements (see references). Please refer to the corresponding repositories to obtain the requirements for these.
At first, download the datasets in section References (Stereo image pairs) as well as the detections and adapt the config files in ./configs according to your desired setup. The file "config_default" will run 2D as well as 3D tracking on the KITTI MOTS validation set. Next, download our pretrained segmentation network and extract it into './external/BB2SegNet'. Before running the main script, run:
python precompute.py -config ./configs/config_default
After this, all necessary information such as segmentations, optical flow, disparity and the corresponding pointcloud should be computed. Now you can run the tracker using:
python main.py -config ./configs/config_default
After the tracker has completed all sequences, results will be evaluated automatically.
We release our results on the MOTS test set cars pedestrians
As well as the detections and segmentations (test-RRC, test-TRCNN, train/val-RRC, train/val-TRCNN).
- RRC Detections: https://github.com/JunaidCS032/MOTBeyondPixels - can be downloaded along with other features in section 'Running the demo script'
- Optical flow/disparity: https://github.com/lmb-freiburg/netdef_models
- Camera pose: https://github.com/raulmur/ORB_SLAM2
- MOTS dataset and TrackRCNN detections & segmentations: https://www.vision.rwth-aachen.de/page/mots
- MOT dataset: http://www.cvlibs.net/datasets/kitti/eval_tracking.php
@article{luiten2020track,
title={Track to reconstruct and reconstruct to track},
author={Luiten, Jonathon and Fischer, Tobias and Leibe, Bastian},
journal={IEEE Robotics and Automation Letters},
volume={5},
number={2},
pages={1803--1810},
year={2020},
publisher={IEEE}
}
MIT License