Long-term Visual Tracking:
This page focuses on watching the state-of-the-art performance for the long-term tracking task (if you are interested in the short-term tracking task, please visit here).
Recent Long-term Trackers
-
LTMU: Kenan Dai, Yunhua Zhang, Dong Wang, Jianhua Li, Huchuan Lu, Xiaoyun Yang.
"High-Performance Long-Term Tracking with Meta-Updater." CVPR (2020). [paper] [code]
VOT2019-LT Winner🌟, VOT20XX-LT Winner🌟
1. This work is an improved version of the VOT2019-LT winner,
[LT_DSE].
2. The baseline version is the VOT20XX-LT winner,
[LTMU_B]. -
Siam R-CNN: Paul Voigtlaender, Jonathon Luiten, Philip H.S. Torr, Bastian Leibe.
"Siam R-CNN: Visual Tracking by Re-Detection." ArXiv (2019). [paper] [code] [project] -
DAL: Yanlin Qian, Alan Lukežič, Matej Kristan, Joni-Kristian Kämäräinen, Jiri Mata
"DAL - A Deep Depth-aware Long-term Tracker" ArXiv (2019). [paper]RGB-D Long-term
-
GlobalTrack: Lianghua Huang, Xin Zhao, Kaiqi Huang.
"GlobalTrack: A Simple and Strong Baseline for Long-term Tracking." AAAI (2020). [paper] [code] -
SPLT: Bin Yan, Haojie Zhao, Dong Wang, Huchuan Lu, Xiaoyun Yang.
"Skimming-Perusal' Tracking: A Framework for Real-Time and Robust Long-Term Tracking." ICCV (2019). [paper] [code] -
flow_MDNet_RPN: Han Wu, Xueyuan Yang, Yong Yang, Guizhong Liu.
"Flow Guided Short-term Trackers with Cascade Detection for Long-term Tracking." ICCVW (2019). [paper] -
OTR: Ugur Kart, Alan Lukezic, Matej Kristan, Joni-Kristian Kamarainen, Jiri Matas.
"Object Tracking by Reconstruction with View-Specific Discriminative Correlation Filters." CVPR (2019). [paper] [code]RGB-D Long-term
-
SiamRPN++: Bo Li, Wei Wu, Qiang Wang, Fangyi Zhang, Junliang Xing, Junjie Yan. "SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks." CVPR (2019). [paper] [code]
-
MBMD: Yunhua Zhang, Dong Wang, Lijun Wang, Jinqing Qi, Huchuan Lu.
"Learning regression and verification networks for long-term visual tracking." Arxiv (2018). [paper] [code] VOT2018-LT Winner🌟 -
MMLT: Hankyeol Lee, Seokeon choi, Changick Kim.
"A Memory Model based on the Siamese Network for Long-term Tracking." ECCVW (2018). [paper] [code] -
FuCoLoT: Alan Lukežič, Luka Čehovin Zajc, Tomáš Vojíř, Jiří Matas and Matej Kristan.
"FuCoLoT - A Fully-Correlational Long-Term Tracker." ACCV (2018). [paper] [code]
Long-term Trackers modified from Short-term Ones
-
SiamDW: Zhipeng Zhang, Houwen Peng.
"Deeper and Wider Siamese Networks for Real-Time Visual Tracking." CVPR (2019). [paper] [code] VOT2019 RGB-D Winner🌟 Denoted as "SiamDW_D" "SiamDW_LT", see the VOT2019 official report [vot2019code] -
DaSiam_LT: Zheng Zhu, Qiang Wang, Bo Li, Wei Wu, Junjie Yan, Weiming Hu.
"Distractor-Aware Siamese Networks for Visual Object Tracking." ECCV (2018). [paper] [code] VOT2018-LT Runner-up🌟
Early Long-term Trackers (before 2018)
-
PTAV: Heng Fan, Haibin Ling.
"Parallel Tracking and Verifying: A Framework for Real-Time and High Accuracy Visual Tracking." ICCV (2017). [paper] [supp] [project] [code] -
EBT: Gao Zhu, Fatih Porikli, Hongdong Li.
"Beyond Local Search: Tracking Objects Everywhere with Instance-Specific Proposals." CVPR (2016). [paper] [exe] -
LCT: Chao Ma, Xiaokang Yang, Chongyang Zhang, Ming-Hsuan Yang.
"Long-term Correlation Tracking." CVPR (2015). [paper] [project] [github] -
MUSTer: Zhibin Hong, Zhe Chen, Chaohui Wang, Xue Mei, Danil Prokhorov, Dacheng Tao.
"MUlti-Store Tracker (MUSTer): a Cognitive Psychology Inspired Approach to Object Tracking." CVPR (2015). [paper] [project] -
CMT: Georg Nebehay, Roman Pflugfelder.
"Clustering of Static-Adaptive Correspondences for Deformable Object Tracking." CVPR (2015). [paper] [project]
[github] -
SPL: James Steven Supančič III, Deva Ramanan.
"Self-paced Learning for Long-term Tracking." CVPR (2013). [paper] [github] -
TLD: Zdenek Kalal, Krystian Mikolajczyk, Jiri Matas.
"Tracking-Learning-Detection." TPAMI (2012). [paper] [project]
Benchmark
-
VOT: . [Visual Object Tracking Challenge]
-
OxUvA: Jack Valmadre, Luca Bertinetto, João F. Henriques, Ran Tao, Andrea Vedaldi, Arnold Smeulders, Philip Torr, Efstratios Gavves.
"Long-term Tracking in the Wild: a Benchmark." ECCV (2018). [paper] [project] -
TLP: Abhinav Moudgil, Vineet Gandhi.
"Long-term Visual Object Tracking Benchmark." ACCV (2018). [paper] [project] -
CDTB: Alan Lukežič, Ugur Kart, Jani Käpylä, Ahmed Durmush, Joni-Kristian Kämäräinen, Jiří Matas, Matej Kristan.
"CDTB: A Color and Depth Visual Object Tracking Dataset and Benchmark." ICCV (2019). [paper] [project]RGB-D Long-term
-
LaSOT: Heng Fan, Liting Lin, Fan Yang, Peng Chu, Ge Deng, Sijia Yu, Hexin Bai, Yong Xu, Chunyuan Liao, Haibin Ling.
"LaSOT: A High-quality Benchmark for Large-scale Single Object Tracking." CVPR (2019). [paper] [project]
The LaSOT dataset is not a typical long-term dataset. But it is a good choice for connecting long-term and short-term trackers. Usually, short-term trackers drift very easily in the long-term datasets since they have no re-detection module. Long-term trackers also achieve unsatisfactory performance in the short-term datasets, since the tested sequences are often very short and the evaluation criterion pay less attention to the re-detection capability (especially VOT' EAO). LaSOT is a large-scale, long-frame dataset with precision and succuess criterion. Thus, it is a good choice if you want to fairly compare the performance of long-term and short-term trackers in one figure/table.
-
UAV20L: Matthias Mueller, Neil Smith and Bernard Ghanem.
"A Benchmark and Simulator for UAV Tracking." ECCV (2016). [paper] [project] [dataset]
All 20 videos of UAV20L have been included in the VOT2018LT dataset.
Measurement&Discussion:
-
Alan Lukežič, Luka Čehovin Zajc, Tomáš Vojíř, Jiří Matas, Matej Krista. "Performance Evaluation Methodology for Long-Term Visual Object Tracking." ArXiv (2019). [paper]
-
Alan Lukežič, Luka Čehovin Zajc, Tomáš Vojíř, Jiří Matas, Matej Kristan. "Now You See Me: Evaluating Performance in Long-term Visual Tracking." ArXiv (2018). [paper]
-
Shyamgopal Karthik, Abhinav Moudgil, Vineet Gandhi. "Exploring 3 R's of Long-term Tracking: Re-detection, Recovery and Reliability." WACV (2020). [paper]
Resources:
-
"Paper, Benchmark, Researchers, Teams" maintained by Qiang Wang:
https://github.com/foolwood/benchmark_results -
"pysot [SiamRPN++, SiamMask, DaSiamRPN, SiamRPN]":
https://github.com/STVIR/pysot -
"pytracking [PrDIMP, SuperDIMP, DIMP, ATOM]":
https://github.com/visionml/pytracking
Benchmark Results:
-
VOT2019-LT/VOT2020-LT:
Tracker F-Score Speed (fps) Paper/Code LTMU (CVPR20) 0.697 13 (RTX 2080Ti) Paper/Code LT_DSE 0.695 N/A N/A CLGS 0.674 N/A N/A SiamDW_LT 0.665 N/A N/A SPLT (ICCV19) 0.587 26 (GTX 1080Ti) Paper/Code mbdet 0.567 N/A N/A SiamRPNsLT 0.556 N/A N/A Siamfcos-LT 0.520 N/A N/A CooSiam 0.508 N/A N/A ASINT 0.505 N/A N/A FuCoLoT 0.411 N/A N/A - Most results are obtained from the original VOT2019_LT report.
-
VOT2018-LT:
Tracker F-Score Speed (fps) Paper/Code LTMU (CVPR20) 0.690 13 (RTX 2080Ti) Paper/Code Siam R-CNN (CVPR20) 0.668 5 (Tesla V100) Paper/Code SiamRPN++ 0.629 35 (Titan XP) Paper/Code SPLT (ICCV19) 0.622 26 (GTX 1080Ti) Paper/Code MBMD (Arxiv) 0.610 4 (GTX 1080Ti) Paper/Code DaSiam_LT (ECCV18) 0.607 110 (TITAN X) Paper/Code - MBMD and DaSiam_LT is the winner and runner-up in the original VOT2018_LT report.
-
OxUvA:
Tracker MaxGM Speed (fps) Paper/Code LTMU (CVPR20) 0.751 13 (RTX 2080Ti) Paper/Code Siam R-CNN (CVPR20) 0.723 5 (Tesla V100) Paper/Code SPLT (ICCV19) 0.622 26 (GTX 1080Ti) Paper/Code GlobalTrack (AAAI20) 0.603 6 (GTX TitanX) Paper/Code MBMD (Arxiv) 0.544 4 (GTX 1080Ti) Paper/Code SiamFC+R (ECCV18) 0.454 52 (Unkown GPU) Paper/Code - OxUvA Leaderboard: https://competitions.codalab.org/competitions/19529#results
- SiamFC+R is the best tracker in the original OxUvA paper.
-
TLP:
Tracker Success Score Speed (fps) Paper/Code LTMU (CVPR20) 0.571 13 (RTX 2080Ti) Paper/Code GlobalTrack (AAAI20) 0.520 6 (GTX TitanX) Paper/Code SPLT (ICCV19) 0.416 26 (GTX 1080Ti) Paper/Code MDNet (CVPR16) 0.372 5 (GTX 1080Ti) Paper/Code - MDNet is the best tracker in the original TLP paper.
-
LaSOT:
Tracker Success Score Speed (fps) Paper/Code Siam R-CNN (CVPR20) 0.648 5 (Tesla V100) Paper/Code PrDiMP50 (CVPR20) 0.598 30 (Unkown GPU) Paper/Code LTMU (CVPR20) 0.572 13 (RTX 2080Ti) Paper/Code DiMP50 (ICCV19) 0.568 43 (GTX 1080) Paper/Code SiamAttn (CVPR20) 0.560 45 (RTX 2080Ti) Paper/Code SiamFC++GoogLeNet (AAAI20) 0.544 90 (RTX 2080Ti) Paper/Code MAML-FCOS (CVPR20) 0.523 42 (NVIDIA P100) Paper/Code GlobalTrack (AAAI20) 0.521 6 (GTX TitanX) Paper/Code ATOM (CVPR19) 0.515 30 (GTX 1080) Paper/Code SiamBAN (CVPR20) 0.514 40 (GTX 1080Ti) Paper/Code SiamCar (CVPR20) 0.507 52 (RTX 2080Ti) Paper/Code SiamRPN++ (CVPR19) 0.496 35 (Titan XP) Paper/Code ROAM++ (CVPR20) 0.447 20 (RTX 2080) Paper/Code SPLT (ICCV19) 0.426 26 (GTX 1080Ti) Paper/Code MDNet (CVPR16) 0.397 5 (GTX 1080Ti) Paper/Code - MDNet is the best tracker in the original LaSOT paper.
All Tracking Datasets:
-
List:
Datasets #videos #total/min/max/average frames Absent Label OTB-2015 100 59K/71/3,872/590 No TC-128 128 55K/71/3,872/429 No NUS-PRO 365 135K/146/5,040/371 No UAV123 123 113K/109/3,085/915 No TB70 70 XXXX No ALOV300++ 315 8.9K/XXXX/XXXX/284 No NfS 100 383K/169/20,665/3,830 No GOT-10k train-10k, val-180, test-180 1.5M No LaSOT 1,400 (I-all-1,400/II-test-280) 3.52M/1,000/11,397/2,506 Yes VOT2019-LT/VOT2020-LT 50 XXXX/XXXX/XXXX/XXXX Yes TLP 50 XXXX/XXXX/XXXX/XXXX No OxUvA 366 (dev-200/test-166) XXXX/XXXX/XXXX/XXXX Yes - OTB-2013 is a subset of OTB-2015.
- UAV-20L has been included in VOT2018-LT/VOT2019-LT/VOT2020-LT.
- VOT2018-LT is a subset of VOT2019-LT/VOT2020-LT. VOT2019-LT and VOT2020-LT are same.