An investigation for RGBD tracking. Hopefully, it can help other researchers become familiar with multi-modal tracking as soon as possible. This repository is started on 27/12/2023, and will keep on updating.
This repository will give a detail investigation of the RGBD tracking community, including the Datasets, Results, and the Methods.
- Datasets
- Results
- Methods
- ...
- RGBD---- A Survey of RGB-Depth Object Tracking. Zhou Ou, Ge Ying, Dawei Zhang*, Zhonglong Zheng. Journal of Computer-Aided Design & Computer Graphics 2024. [Paper]
- RGBD/T ---- Multi-modal visual tracking: Review and experimental comparison. Zhang, Pengyu, Dong Wang*, and Huchuan Lu. Computational Visual Media 2024. [Paper]
- RGBD---- Rgbd object tracking: An in-depth review. Jinyu Yang, Zhe Li, Song Yan, Feng Zheng*, Aleš Leonardis, Joni-Kristian Kämäräinen, Ling Shao. Arxiv 2022. [Paper]
Real Data
| Dataset | Publish | GitHub | Introduction |
|---|---|---|---|
| D2CUBE | CVRP'2023 | D2CUBE | Resource-Efficient RGBD Aerial Tracking |
| ARKittrack | CVPR'2023 | ARKittrack | ARKitTrack: A New Diverse Dataset for Tracking Using Mobile RGB-D Data |
| RGBD1K | AAAI'2023 | RGBD1K | RGBD1K: A Large-Scale Dataset and Benchmark for RGB-D Object Tracking |
| VOT-RGBD2022 | VOT Community | VOT-RGBD2022 | The Tenth Visual Object Tracking VOT2022 Challenge Results |
| DepthTrack | ICCV'2021 | DepthTrack | DepthTrack: Unveiling the Power of RGBD Tracking |
| CDTB | ICCV'2019 | CDTB | CDTB: A Color and Depth Visual Object Tracking Dataset and Benchmark |
| STC | ICCV'2019 | STC code:TZYD | Robust Fusion of Color and Depth Data for RGB-D Target Tracking Using Adaptive Range-Invariant Depth Models and Spatio-Temporal Consistency Constraints |
| PTB | ICCV'2013 | PTB | Tracking Revisited using RGBD Camera: Unified Benchmark and Baselines |
| BoBoT | - | BoBoT | BoBot - Bonn benchmark on tracking |
| CDTB | DepthTrack | VOT-RGBD2022 | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Methods | Venue | Speed | Pr | Re | F-score | Pr | Re | F-score | A | R | EAO |
| UBPT | IEEE Sensor Journal'2024 | 61.5 | 62.0 | 61.7 | 82.0 | 87.1 | 72.1 | ||||
| DepthRefiner | ICME'2024 | 32/A100 | 66.9 | 68.4 | 67.7 | 51.3 | 50.7 | 51.0 | 79.7 | 73.3 | 60.3 |
| TABBTrack | PR'2024 | 27/RTX3090 | 72.1 | 72.2 | 72.1 | 62.2 | 61.5 | 61.8 | 82.1 | 87.4 | 72.2 |
| AMATrack | TIM'2024 | 73/RTX3070 | 73.2 | 78.6 | 75.8 | 62.9 | 60.7 | 61.8 | |||
| MixRGBX | Neurocomputing'2024 | 72.8 | 81.6 | 76.9 | 59.3 | 60.9 | 60.1 | ||||
| OneTrack | CVPR'2024 | 60.7 | 60.4 | 60.9 | 87.2 | 81.9 | 72.7 | ||||
| UnTrack | CVPR'2024 | 61.3 | 61.0 | 61.2 | 87.1 | 81.5 | 72.1 | ||||
| SDSTrack | CVPR'2024 | 61.9 | 60.9 | 61.4 | 88.3 | 81.2 | 72.8 | ||||
| XTrack | Arxiv'2024 | 59.8 | 59.7 | 59.7 | 86.5 | 81.2 | 71.4 | ||||
| Seqtrackv2 | Arxiv'2024 | 62.9 | 63.4 | 63.2 | 81.9 | 91.8 | 75.5 | ||||
| MINet | IVC'2024 | 60.3 | 60.5 | 60.4 | 81.6 | 87.7 | 72.3 | ||||
| KSTrack | TCSVT'2024 | 60.0 | 57.4 | 58.7 | |||||||
| CDAAT | SPL'2024 | 59.5 | 66.5 | 73.7 | 69.9 | 57.8 | 60.3 | 59.0 | |||
| FECD | PRL'2024 | 63.7 | 62.4 | 63.0 | 57.8 | 60.3 | 59.0 | ||||
| SSLTrack | PR'2024 | 31.4 | 65.0 | 62.0 | 63.5 | 56.5 | 49.1 | 52.5 | |||
| VADT | ICASSP'2024 | 60.6 | 60.3 | 61.0 | 81.6 | 87.3 | 72.1 | ||||
| ViPT | CVPR'2023 | 59.2 | 59.6 | 59.4 | 87.1 | 81.5 | 72.1 | ||||
| FDAFT | PRCV'2023 | 62.5 | 61.5 | 62.0 | |||||||
| SPT | AAAI'2023 | 25.3 | 65.4 | 72.6 | 68.8 | 52.7 | 54.9 | 53.8 | |||
| HMAD | ACMMMA'2023 | 50.0 | 62.6 | 59.7 | 61.1 | ||||||
| DMTracker | ECCVW'2022 | 66.2 | 65.8 | 66.0 | 61.9 | 59.7 | 60.8 | ||||
| ProTrack | ACMMM'2022 | 74.7 | 76.7 | 65.6 | 58.3 | 57.3 | 57.8 | ||||
| DeT | ICCV'2021 | 67.4 | 64.2 | 65.7 | 56.0 | 50.6 | 53.2 | ||||
| TSDM | ICPR'2021 | 53.5 | |||||||||
| SiamOC | ICSP'2021 | 41.1 | 34.6 | 37.6 | |||||||
| DAL | ICPR'2021 | 20 | 61.8 | ||||||||
| RGBD1K | D2CUBE | ARKittrack | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Methods | Venue | Speed | Pr | Re | F-score | Pr | Re | F-score | Pr | Re | F-score |
| DepthRefiner | ICME'2024 | 32/A100 | 50.0 | 52.9 | 51.6 | 51.0 | 47.8 | 49.3 | |||
| TABBTrack | PR'2024 | 27/RTX3090 | 51.0 | 47.8 | 49.3 | ||||||
| CDAAT | SPL'2024 | 59.5 | 54.9 | 58.3 | 56.6 | ||||||
| SSLTrack | PR'2024 | 57.0 | 47.8 | 52.0 | |||||||
| EMT | CVPR'2023 | 120.3 | 65.3 | 60.9 | 63.0 | ||||||
| RGBD1K | AAAI'2023 | 25.3 | 54.5 | 57.8 | 56.1 | ||||||
| HMAD | ACMMMA'2023 | 50.0 | 57.3 | 55.2 | 56.2 | ||||||
| DeT | ICCV'2021 | 60.8 | 58.7 | 59.7 | 42.8 | 40.5 | 41.6 | ||||
| DAL | ICPR'2021 | 52.9 | 56.5 | 54.7 | 44.6 | 32.9 | 37.8 | ||||
| TSDM | ICPR'2021 | 52.1 | 49.2 | 50.6 | 38.9 | 29.2 | 33.4 | ||||
| STC | PTB | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Methods | Venue | Speed | PR | SR | Human | Animal | Rigid | Large | Small | Slow | Fast | Occ. | No-Occ. | Passive | Active |
| KSTrack | TCSVT'2024 | 77.3 | 84.9 | 83.6 | 79.8 | 83.3 | 82.6 | 81.3 | 73.4 | 96.8 | 80.3 | 82.2 | |||
| SSLTrack | PR'2024 | 64.0 | |||||||||||||
| FECD | PRL'2024 | 63.0 | 65.0 | 85.0 | 88.0 | 75.0 | 80.0 | 88.0 | 73.0 | 65.0 | 94.0 | 89.0 | 73.0 | ||
| RGBD1K | AAAI'2023 | 25.3 | 67.0 | ||||||||||||
| DMTracker | ECCVW'2022 | 63.0 | |||||||||||||
| TSDM | ICPR'2021 | 71.0 | 85.0 | 86.0 | 77.0 | 81.0 | 87.0 | 76.0 | 69.0 | 94.0 | 84.0 | 78.0 | |||
| 3s-RGBD | Neurocomputing'2021 | 59.0 | 49.0 | 77.0 | 68.0 | 81.0 | 76.0 | 77.0 | 81.0 | 75.0 | 71.0 | 85.0 | 85.0 | 74.0 | |
| DAL | ICPR'2021 | 20 | 85.0 | 64.0 | 78.0 | 86.0 | 81.0 | 76.0 | 84.0 | 83.0 | 80.0 | 72.0 | 93.0 | 78.0 | 82.0 |
| WCO | IEEE Sensors Journal'2020 | 78.0 | 67.0 | 80.0 | 76.0 | 75.0 | 78.0 | 73.0 | 66.0 | 86.0 | 85.0 | 72.0 | |||
| RF-CFF | Applied Soft Computing Journal'2020 | 62.0 | 79.0 | 78.0 | 69.0 | 73.0 | 81.0 | 68.0 | 57.0 | 91.0 | 80.0 | 68.0 | |||
| CF-RGBD | Engineering Applications of Artificial Intelligence'2020 | 61.0 | 75.0 | 80.0 | 71.0 | 71.0 | 79.0 | 68.0 | 56.0 | 91.0 | 80.0 | 67.0 | |||
| CA3DMS | TMM'2019 | 63 | 64.0 | 73.0 | 81.0 | 73.0 | 72.0 | 80.0 | 69.0 | 61.0 | 88.0 | 83.0 | 68.0 | ||
| Depth-CCF | GSKI'2019 | 70.0 | 65.0 | 79.0 | 71.0 | 73.0 | 78.0 | 70.0 | 64.0 | 84.0 | 84.0 | 67.0 | |||
| H-FCN | INFFUS'2019 | 19.47 | 81.0 | 74.0 | 80.0 | 82.0 | 77.0 | 78.0 | 74.0 | 83.0 | 87.0 | 80.0 | 78.0 | ||
| OTR | CVPR'2019 | 59.0 | 49.0 | 77.0 | 68.0 | 81.0 | 76.0 | 77.0 | 81.0 | 75.0 | 71.0 | 85.0 | 85.0 | 74.0 | |
| RGBD-OD | CIS'2019 | 72.0 | 71.0 | 73.0 | 74.0 | 71.0 | 76.0 | 70.0 | 65.0 | 82.0 | 77.0 | 70.0 | |||
| HST | CCIFER'2019 | 66.0 | 62.0 | 77.0 | 69.0 | 69.0 | 74.0 | 68.0 | 62.0 | 79.0 | 78.0 | 66.0 | |||
| ECO_TA | IEEE Sensors Journal'2019 | 13.1 | 77.0 | 65.0 | 79.0 | 77.0 | 74.0 | 79.0 | 74.0 | 68.0 | 85.0 | 84.0 | 72.0 | ||
| GFL | Complexity'2019 | 20.74 | 82.0 | 75.0 | 78.0 | 81.0 | 74.0 | 82.0 | 73.0 | 81.0 | 84.0 | 79.0 | 68.0 | ||
| DM-DCF | ICPR'2018 | 8.3 | 76.0 | 58.0 | 77.0 | 72.0 | 73.0 | 75.0 | 72.0 | 69.0 | 78.0 | 82.0 | 69.0 | ||
| CSRDCF_RGBD++ | ECCVW'2018 | 77.0 | 65.0 | 76.0 | 75.0 | 73.0 | 80.0 | 72.0 | 70.0 | 79.0 | 79.0 | 72.0 | |||
| MMDFF | Complexity'2018 | 83.0 | 86.0 | 85.0 | 85.0 | 86.0 | 82.0 | 83.0 | 87.0 | 87.0 | 82.0 | 83.0 | |||
| OAPCF | IEEE Access'2018 | 17 | 70.0 | 66.0 | 68.0 | 69.0 | 70.0 | 75.0 | 67.0 | 73.0 | 76.0 | 71.0 | 69.0 | ||
| KCFDF | ICONIP'2017 | 10.49 | 45.0 | 72.0 | 75.0 | 55.0 | 67.0 | 73.0 | 57.0 | 43.0 | 87.0 | 69.0 | 59.0 | ||
| DBM | Sensors'2017 | 0.1 | 80.1 | 72.9 | 82.3 | 77.5 | 81.2 | 82.6 | |||||||
| DLS | ICPR'2016 | 77.0 | 69.0 | 73.0 | 80.0 | 70.0 | 73.0 | 74.0 | 66.0 | 85.0 | 72.0 | 75.0 | |||
| OAPF | CVIU'2016 | 0.9 | 64.2 | 84.8 | 77.2 | 72.7 | 73.4 | 85.1 | 68.4 | 64.4 | 85.1 | 77.7 | 71.4 | ||
| 3D-T | CVPR'2016 | 81.0 | 64.0 | 73.0 | 80.0 | 71.0 | 75.0 | 75.0 | 73.0 | 78.0 | 79.0 | 73.0 | |||
| DOHR | FSKD'2016 | 45.0 | 49.0 | 42.0 | 48.0 | 42.0 | 50.0 | 43.0 | 38.0 | 54.0 | 54.0 | 41.0 | |||
Questions
If you have any questions, please contact zhangyong_tang_jnu@163.com, and wechat: Tzy18861871359 is also welcomed.