Awesome Multiple object Tracking: Awesome

A curated list of multi-object-tracking and related area resources. It only contains online methods. 中文版更为详细,具体查看仓库根目录下的README-zh.md文件。

Contents

Review papers

Multiple Object Tracking: A Literature Review [paper]

Deep Learning in Video Multi-Object Tracking: A Survey [paper]

Tracking the Trackers: An Analysis of the State of the Art in Multiple Object Tracking [paper]

Machine Learning Methods for Data Association in Multi-Object Tracking [paper]

MOTChallenge: A Benchmark for Single-camera Multiple Target Tracking [paper] new paper for new MOT researcher

Algorithm papers

2022

MeMOT: MeMOT: Multi-Object Tracking with Memory [[code]] [paper] CVPR2022 oral

TAO-OW: Opening up Open-World Tracking [paper] CVPR2022 oral

UTT: Unified Transformer Tracker for Object Trackin [code] [paper] CVPR2022

GTT: Global Tracking Transformers [code] [paper]

OC-SORT: Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking [code] [paper]

StrongSORT: StrongSORT: Make DeepSORT Great Again [[code]] [paper]

MAA: Modelling Ambiguous Assignments for Multi-Person Tracking in Crowds [[code]] [paper]

CrowdTrack: On the Performance of Crowd-Specific Detectors in Multi-Pedestrian Tracking [[code]] [paper]

2021

ByteTrack: ByteTrack: Multi-Object Tracking by Associating Every Detection Box [code] [paper] new SOTA

PermaTrack: Learning to Track with Object Permanence [code] [paper] ICCV2021

TMOH: Improving Multiple Pedestrian Tracking by Track Management and Occlusion Handling [code] [paper] CVPR2021

SOTMOT: Improving Multiple Object Tracking with Single Object Tracking [code] [paper] CVPR2021

LPC_MOT: Learning a Proposal Classifier for Multiple Object Tracking [code] [paper] CVPR2021

MTP: Discriminative Appearance Modeling with Multi-track Pooling for Real-time Multi-object Tracking [code] [paper] CVPR2021

TADAM: Online Multiple Object Tracking with Cross-Task Synergy [code] [paper] CVPR2021

RelationTrack: RelationTrack: Relation-aware Multiple Object Tracking with Decoupled Representation [[code]] [paper]

MOTR: MOTR: End-to-End Multiple-Object Tracking with TRansformer [code] [paper]

OMC: One More Check: Making "Fake Background" Be Tracked Again [code] [paper] AAAI2022

QDTrack: Quasi-Dense Similarity Learning for Multiple Object Tracking [code] [paper] CVPR2021

SiamMOT: SiamMOT: Siamese Multi-Object Tracking [code] [paper] CVPR2021

GMTracker: Learnable Graph Matching: Incorporating Graph Partitioning with Deep Feature Learning for Multiple Object Tracking [code] [paper] CVPR2021

ArTIST: Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking [paper] CVPR2021

CorrTracker/TLR: Multiple Object Tracking with Correlation Learning [code] [paper] CVPR2021

TransMOT:Spatial-Temporal Graph Transformer for Multiple Object Tracking [code] [paper]

TransCenter: TransCenter: Transformers with Dense Queries for Multiple-Object Tracking [code] [paper]

GCNet: Global Correlation Network: End-to-End Joint Multi-Object Detection and Tracking [code] [paper]

TraDes: Track to Detect and Segment: An Online Multi-Object Tracker [code] [paper] CVPR2021

DEFT: DEFT: Detection Embeddings for Tracking [code] [paper]

TrackMPNN: TrackMPNN: A Message Passing Graph Neural Architecture for Multi-Object Tracking [code] [paper]

TrackFormer: TrackFormer: Multi-Object Tracking with Transformers [[code]] [paper]

2020

TransTrack: TransTrack: Multiple-Object Tracking with Transformer [code] [paper]

TPAGT: Tracklets Predicting Based Adaptive Graph Tracking [paper] original FGAGT

MLT: Multiplex Labeling Graph for Near-Online Tracking in Crowded Scenes [paper]

GSDT: Joint Object Detection and Multi-Object Tracking with Graph Neural Networks [paper]

SMOT: SMOT: Single-Shot Multi Object Tracking [paper]

CSTrack: Rethinking the competition between detection and ReID in Multi-Object Tracking [code][paper]

MAT: MAT: Motion-Aware Multi-Object Tracking [paper]

UnsupTrack: Simple Unsupervised Multi-Object Tracking [paper]

FairMOT: FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking [code][paper] a new version of FairMOT, compared with new method like CTracker

DMM-Net: Simultaneous Detection and Tracking with Motion Modelling for Multiple Object Tracking [code][paper]

SoDA: SoDA: Multi-Object Tracking with Soft Data Association [[code]][paper]

CTracker: Chained-Tracker: Chaining Paired Attentive Regression Results for End-to-End Joint Multiple-Object Detection and Tracking [code][paper]

MPNTracker: Learning a Neural Solver for Multiple Object Tracking [code][paper]

UMA: A Unified Object Motion and Affinity Model for Online Multi-Object Tracking [code][paper]

RetinaTrack: Online Single Stage Joint Detection and Tracking [[code]][paper]

FairMOT: A Simple Baseline for Multi-Object Tracking [code][paper]

TubeTK: TubeTK: Adopting Tubes to Track Multi-Object in a One-Step Training Model [code][paper]

CenterTrack: Tracking Objects as Points [code][paper]

Lif_T: Lifted Disjoint Paths with Application in Multiple Object Tracking [code][paper]

PointTrack: Segment as points for efficient online multi-object tracking and segmentation [code][paper]

PointTrack++: PointTrack++ for Effective Online Multi-Object Tracking and Segmentation [code][paper]

FFT: Multiple Object Tracking by Flowing and Fusing [paper]

MIFT: Refinements in Motion and Appearance for Online Multi-Object Tracking [code][paper]

EDA_GNN: Graph Neural Based End-to-end Data Association Framework for Online Multiple-Object Tracking [code][paper]

GNMOT: Graph Networks for Multiple Object Tracking [code][paper]

2019

Tracktor/Tracktor++: Tracking without bells and whistles [code][paper]

DeepMOT: How To Train Your Deep Multi-Object Tracker [code][paper]

JDE: Towards Real-Time Multi-Object Tracking [code][paper]

MOTS: MOTS: Multi-Object Tracking and Segmentation[paper]

FANTrack: FANTrack: 3D Multi-Object Tracking with Feature Association Network [code][paper]

FAMNet: Joint Learning of Feature, Affinity and Multi-dimensional Assignment for Online Multiple Object Tracking[paper]

2018

DeepCC: Features for Multi-Target Multi-Camera Tracking and Re-Identification [paper]

SADF: Online Multi-Object Tracking with Historical Appearance Matching and Scene Adaptive Detection Filtering [paper]

DAN: Deep Affinity Network for Multiple Object Tracking [code][paper]

DMAN: Online Multi-Object Tracking with Dual Matching Attention Networks [code][paper]

MOTBeyondPixels: Beyond Pixels: Leveraging Geometry and Shape Cues for Online Multi-Object Tracking [code][paper]

MOTDT: Real-time Multiple People Tracking with Deeply Learned Candidate Selection and Person Re-Identification [code][paper]

DetTA: Detection-Tracking for Efficient Person Analysis: The DetTA Pipeline [code][paper]

V-IOU: Extending IOU Based Multi-Object Tracking by Visual Information [code][paper]

2017

DeepSORT: Simple Online and Realtime Tracking with a Deep Association Metric [code][paper]

NMGC-MOT: Non-Markovian Globally Consistent Multi-Object Tracking [code][paper]

IOUTracker: High-Speed tracking-by-detection without using image information [code][paper]

RNN_LSTM: Online Multi-Target Tracking Using Recurrent Neural Networks [code][paper]

D2T: Detect to Track and Track to Detect [code][paper]

RCMSS: Online multi-object tracking via robust collaborative model and sample selection [paper]

towards-reid-tracking: Towards a Principled Integration of Multi-Camera Re-Identification and Tracking through Optimal Bayes Filters [code][paper]

CIWT: Combined image-and world-space tracking in traffic scenes [code][paper]

2016

SORT: Simple online and realtime tracking [code][paper]

POI: POI: Multiple Object Tracking with High Performance Detection and Appearance Feature [code]

Datasets

Surveillance Scenarios

PETS 2009 Benchmark Data [url]
MOT Challenge [url]
UA-DETRAC [url]
WILDTRACK [url]
NVIDIA AI CITY Challenge [url]
VisDrone [url]
JTA Dataset [url]
Path Track [url]
TAO [url]
GMOT40 [url]

Driving Scenarios

KITTI-Tracking [url]
APOLLOSCAPE [url]
APOLLO MOTS [url]
Omni-MOT [url]
BDD100K [url]
Waymo [url]

Metrics

metric formula
accuracy $ Accuracy = {{TP + TN} \over {TP + TN + FP + FN}} $
recall $ Recall = {TP \over {TP + FN}} = TPR$
precision $ Precision = {TP \over {TP + FP}} $
MA $ MA = {FN \over {TP + FN}} $
FA $ FA = {FP \over {TP + FP}} $
MOTA $MOTA = 1 - {\sum_t(FN + FP + IDs)\over \sum_t gt}$
MOTP $ MOTP = {\sum_{t,i}d_t^i \over \sum_tc_t }$
IDP $ IDP = {IDTP \over {IDTP + IDFP}} $
IDR $ IDR = {IDTP \over {IDTP + IDFN}} $
IDF1 $ IDF1 = {2 \over {{1 \over IDP} + {1 \over IDR}}} = {2IDTP \over {2IDTP + IDFP + IDFN}} $

Evaluation code

Benchmark Results

MOT16

Rank Model MOTA Paper Year
1 FairMOT 68.7 A Simple Baseline for Multi-Object Tracking 2020
2 JDE 64.4 Towards Real-Time Multi-Object Tracking 2019
3 Lif_T 61.3 Lifted Disjoint Paths with Application in Multiple Object Tracking 2020
4 MPNTrack 58.6 Learning a Neural Solver for Multiple Object Tracking 2020
5 DeepMOT-Tracktor 54.8 How To Train Your Deep Multi-Object Tracker 2019
6 TNT 49.2 Exploit the Connectivity: Multi-Object Tracking with TrackletNet 2018
7 GCRA 48.2 Trajectory Factory: Tracklet Cleaving and Re-connection by Deep Siamese Bi-GRU for Multiple Object Tracking 2018
8 FWT 47.8 Fusion of Head and Full-Body Detectors for Multi-Object Tracking 2017
9 MOTDT 47.6 Real-time Multiple People Tracking with Deeply Learned Candidate Selection and Person Re-Identification 2018
10 NOMT 46.4 Near-Online Multi-target Tracking with Aggregated Local Flow Descriptor 2015
11 DMMOT 46.1 Online Multi-Object Tracking with Dual Matching Attention Networks 2019

MOT17

Rank Model MOTA Paper Year
1 FairMOT 67.5 A Simple Baseline for Multi-Object Tracking 2020
2 Lif_T 60.5 Lifted Disjoint Paths with Application in Multiple Object Tracking 2020
3 MPNTrack 58.8 Learning a Neural Solver for Multiple Object Tracking 2020
4 DeepMOT 53.7 How To Train Your Deep Multi-Object Tracker 2019
5 JBNOT 52.6 Multiple People Tracking using Body and Joint Detections 2019
6 TNT 51.9 Exploit the Connectivity: Multi-Object Tracking with TrackletNet 2018
7 FWT 51.3 Fusion of Head and Full-Body Detectors for Multi-Object Tracking 2017
8 MOTDT17 50.9 Real-time Multiple People Tracking with Deeply Learned Candidate Selection and Person Re-Identification 2018

MOT20

Rank Model MOTA Paper Year
1 FairMOT 61.8 A Simple Baseline for Multi-Object Tracking 2020
2 UnsupTrack 53.6 Simple Unsupervised Multi-Object Tracking 2020

Toolbox

mmtracking: OpenMMLab Video Perception Toolbox. It supports Single Object Tracking (SOT), Multiple Object Tracking (MOT), Video Object Detection (VID) with a unified framework.

Github DOC

Course

link is a good course about multiple object tracking. The course is offered as a Massive Open Online Course (MOOC) on edX.