Collection of papers, datasets, code and other resources for object detection and tracking using deep learning
- Papers
- Datasets
- Code
- Collections
- Tutorials
Papers
Static Detection
Region Proposal
- Scalable Object Detection Using Deep Neural Networks [cvpr14] [pdf] [notes]
- Selective Search for Object Recognition [ijcv2013] [pdf] [notes]
RCNN
- Faster R-CNN Towards Real-Time Object Detection with Region Proposal Networks [tpami17] [pdf] [notes]
- RFCN - Object Detection via Region-based Fully Convolutional Networks [nips16] [Microsoft Research] [pdf] [notes]
- Mask R-CNN [iccv17] [Facebook AI Research] [pdf] [notes] [arxiv] [code (keras)] [code (tensorflow)]
YOLO
- You Only Look Once Unified, Real-Time Object Detection [ax1605] [pdf] [notes]
- YOLO9000 Better, Faster, Stronger [ax1612] [pdf] [notes]
- YOLOv3 An Incremental Improvement [ax1804] [pdf] [notes]
SSD
- SSD Single Shot MultiBox Detector [ax1612/eccv16] [pdf] [notes]
- DSSD Deconvolutional Single Shot Detector [ax1701] [pdf] [notes]
RetinaNet
- Feature Pyramid Networks for Object Detection [ax1704] [pdf] [notes]
- Focal Loss for Dense Object Detection [ax180207/iccv17] [pdf] [notes]
Misc
- OverFeat Integrated Recognition, Localization and Detection using Convolutional Networks [ax1402/iclr14] [pdf] [notes]
- LSDA Large scale detection through adaptation [ax1411/nips14] [pdf] [notes]
- Acquisition of Localization Confidence for Accurate Object Detection [ax1807/eccv18] [pdf] [notes] [code]
Video Detection
Tubelet
- Object Detection from Video Tubelets with Convolutional Neural Networks [cvpr16] [pdf] [notes]
- Object Detection in Videos with Tubelet Proposal Networks [ax1704/cvpr17] [pdf] [notes]
FGFA
- Deep Feature Flow for Video Recognition [cvpr17] [Microsoft Research] [pdf] [arxiv] [code]
- Flow-Guided Feature Aggregation for Video Object Detection [ax1708/iccv17] [pdf] [notes]
- Towards High Performance Video Object Detection [ax1711] [Microsoft] [pdf] [notes]
RNN
- Online Video Object Detection using Association LSTM [iccv17] [pdf] [notes]
- Context Matters Refining Object Detection in Video with Recurrent Neural Networks [bmvc16] [pdf] [notes]
Multi Object Tracking
Deep Learning
- Tracking The Untrackable: Learning To Track Multiple Cues with Long-Term Dependencies [ax1704/iccv17] [Stanford] [pdf] [arxiv] [project], [notes]
Reinforcement Learning
- Learning to Track: Online Multi-object Tracking by Decision Making [iccv15] [Stanford] [pdf] [code (matlab)] [project] [notes]
Network Flow
- Near-Online Multi-target Tracking with Aggregated Local Flow Descriptor [iccv15] [NEC Labs] [pdf] [author] [notes]
- Deep Network Flow for Multi-Object Tracking [cvpr17] [NEC Labs] [pdf] [supplementary] [notes]
Graph Optimization
- A Multi-cut Formulation for Joint Segmentation and Tracking of Multiple Objects [ax1607] [highest MT on MOT2015] [University of Freiburg, Germany] [pdf] [arxiv] [author] [notes]
Baseline
- Simple Online and Realtime Tracking [icip16] [pdf] [notes] [code])
- High-Speed Tracking-by-Detection Without Using Image Information [avss17] [pdf] [notes] [code]
Single Object Tracking
Reinforcement Learning
- Deep Reinforcement Learning for Visual Object Tracking in Videos [ax1704] [USC-Santa Barbara, Samsung Research] [pdf] [arxiv] [author] [notes]
- Visual Tracking by Reinforced Decision Making
[ax1702] [Seoul National University, Chung-Ang University]
[pdf]
[arxiv]
[author]
[notes] - Action-Decision Networks for Visual Tracking with Deep Reinforcement Learning [cvpr17] [Seoul National University] [pdf] [supplementary] [project] [notes] [code]
- End-to-end Active Object Tracking via Reinforcement Learning [ax1705] [Peking University, Tencent AI Lab] [pdf] [arxiv]
Siamese
- Fully-Convolutional Siamese Networks for Object Tracking [eccv16] [pdf] [project] [notes]
- High Performance Visual Tracking with Siamese Region Proposal Network [cvpr18] [pdf] [author] [notes]
Misc
Deep Learning
Synthetic Gradients
- Decoupled Neural Interfaces using Synthetic Gradients [ax1608] [pdf] [notes]
- Understanding Synthetic Gradients and Decoupled Neural Interfaces [ax1703] [pdf] [notes]
Unsupervised Learning
Interpolation
- Video Frame Interpolation via Adaptive Convolution [cvpr17 / iccv17] [pdf (cvpr17)] [pdf (iccv17)] [ppt]
Autoencoder
Variational
- beta-VAE Learning Basic Visual Concepts with a Constrained Variational Framework [iclr17] [pdf] [notes]
- Disentangling by Factorising [ax1806] [pdf] [notes]
Datasets
Multi Object Tracking
- IDOT
- UA-DETRAC Benchmark Suite
- GRAM Road-Traffic Monitoring
- Stanford Drone Dataset
- Ko-PER Intersection Dataset
- TRANCOS
- Urban Tracker
- DARPA VIVID / PETS 2005 [Non stationary camera]
- KIT-AKS [No ground truth]
- CBCL StreetScenes Challenge Framework [No top down viewpoint]
- MOT 2015 [mostly street level viewpoint]
- MOT 2016 [mostly street level viewpoint]
- MOT 2017 [mostly street level viewpoint]
- PETS 2009 [No vehicles]
- PETS 2017 [Low density] [mostly pedestrians]
- KITTI Tracking Dataset [No top down viewpoint] [non stationary camera]
- The WILDTRACK Seven-Camera HD Dataset [pedestrian detection and tracking]
- 3D Traffic Scene Understanding from Movable Platforms [intersection traffic] [stereo setup] [moving camera]
- LOST : Longterm Observation of Scenes with Tracks [top down and street level viewpoint] [no ground truth]
Single Object Tracking
- TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild [eccv18]
- LaSOT: Large-scale Single Object Tracking [cvpr19]
- Need for speed: A benchmark for higher frame rate object tracking [iccv17]
- Long-term Tracking in the Wild A Benchmark [eccv18]
- UAV123: A benchmark and simulator for UAV tracking [eccv16] [project]
- Sim4CV A Photo-Realistic Simulator for Computer Vision Applications [ijcv18]
Video Detection
Video Understanding / Activity Recognition
Static Detection
- PASCAL Visual Object Classes
- A Large-Scale Dataset for Vehicle Re-Identification in the Wild [cvpr19]
- Object Detection-based annotations for some frames of the VIRAT dataset
Animals
- Wildlife Image and Localization Dataset (species and bounding box labels) [wacv18]
- Stanford Dogs Dataset [cvpr11]
- Oxford-IIIT Pet Dataset [cvpr12]
- Caltech-UCSD Birds 200 [rough segmentation] [attributes]
- Gold Standard Snapshot Serengeti Bounding Box Coordinates
Boundary Detection
Static Segmentation
- COCO - Common Objects in Context
- Open Images
- ADE20K [cvpr17]
- SYNTHIA [cvpr16]
- UC Berkeley Computer Vision Group - Contour Detection and Image Segmentation
Video Segmentation
- DAVIS: Densely Annotated VIdeo Segmentation
- Mapillary Vistas Dataset [street scenes] [semi-free]
Classification
Optical Flow
Code
Multi Object Tracking
- Globally-optimal greedy algorithms for tracking a variable number of objects [cvpr11] [matlab] [author]
- Continuous Energy Minimization for Multitarget Tracking [cvpr11 / iccv11 / tpami 2014] [matlab]
- Discrete-Continuous Energy Minimization for Multi-Target Tracking [cvpr12] [matlab] [project]
- The way they move: Tracking multiple targets with similar appearance [iccv13] [matlab]
- 3D Traffic Scene Understanding from Movable Platforms [2d_tracking] [pami14/kit13/iccv13/nips11] [c++/matlab]
- Multiple target tracking based on undirected hierarchical relation hypergraph [cvpr14] [C++] [author]
- Robust online multi-object tracking based on tracklet confidence and online discriminative appearance learning [cvpr14] [matlab] (project)
- Learning to Track: Online Multi-Object Tracking by Decision Making [iccv15] [matlab]
- Joint Tracking and Segmentation of Multiple Targets [cvpr15] [matlab]
- Multiple Hypothesis Tracking Revisited [iccv15] [highest MT on MOT2015 among open source trackers] [matlab]
- Simple Online and Realtime Tracking [icip 2016] [python]
- Deep SORT : Simple Online Realtime Tracking with a Deep Association Metric [icip17] [python]
- Combined Image- and World-Space Tracking in Traffic Scenes [icra 2017] [c++]
- High-Speed Tracking-by-Detection Without Using Image Information [avss17] [python]
- Tracking by Animation: Unsupervised Learning of Multi-Object Attentive Trackers [cvpr19] [python/c++/pytorch(TBA)]
- TrackR-CNN - Multi-Object Tracking and Segmentation [cvpr19] [tensorflow]
Single Object Tracking
- A collection of common tracking algorithms (2003-2012) [c++/matlab]
- SenseTime Research platform for single object tracking, implementing algorithms like SiamRPN and SiamMask [pytorch]
- In Defense of Color-based Model-free Tracking [cvpr15] [c++]
- Hierarchical Convolutional Features for Visual Tracking [iccv15] [matlab]
- Visual Tracking with Fully Convolutional Networks [iccv15] [matlab]
- DeepTracking: Seeing Beyond Seeing Using Recurrent Neural Networks [aaai16] [torch 7]
- Learning Multi-Domain Convolutional Neural Networks for Visual Tracking [cvpr16] [vot2015 winner] [matlab/matconvnet]
- Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking [eccv 2016] [matlab]
- Fully-Convolutional Siamese Networks for Object Tracking [eccvw 2016] [matlab/matconvnet] [project] [pytorch] [pytorch (only training)]
- DCFNet: Discriminant Correlation Filters Network for Visual Tracking [ax1704] [matlab/matconvnet] [pytorch]
- End-to-end representation learning for Correlation Filter based tracking [cvpr17] [matlab/matconvnet] [tensorflow/inference_only] [project]
- A simplified PyTorch implementation of Siamese networks for tracking: SiamFC, SiamRPN, SiamRPN++, SiamVGG, SiamDW, SiamRPN-VGG [pytorch]
- RATM: Recurrent Attentive Tracking Model [cvprw17] [python]
- ROLO : Spatially Supervised Recurrent Convolutional Neural Networks for Visual Object Tracking [iscas 2017] [tensorfow]
- ECO: Efficient Convolution Operators for Tracking [cvpr17] https://github.com/martin-danelljan/ECO[matlab] python/cuda pytorch
- Action-Decision Networks for Visual Tracking with Deep Reinforcement Learning [cvpr17] [tensorflow]
- Detect to Track and Track to Detect [iccv17] [matlab]
- High Performance Visual Tracking with Siamese Region Proposal Network [cvpr18] [pytorch] [pytorch] [pytorch/no_train] [pytorch]
- Distractor-aware Siamese Networks for Visual Object Tracking [eccv18] [vot18 winner] [pytorch]
- Fast Online Object Tracking and Segmentation: A Unifying Approach (SiamMask) [cvpr19] [pytorch] [project]
- PyTracking: A general python framework for training and running visual object trackers, based on PyTorch [ECO/ATOM/DiMP] [cvpr17/cvpr19/iccv19] [pytorch]
GUI Application / Large Scale Tracking / Animals
- BioTracker An Open-Source Computer Vision Framework for Visual Animal Tracking[opencv/c++]
- Tracktor: Image‐based automated tracking of animal movement and behaviour[opencv/c++]
- MARGO (Massively Automated Real-time GUI for Object-tracking), a platform for high-throughput ethology[matlab]
- idtracker.ai: Tracking all individuals in large collectives of unmarked animals [tensorflow] [project]
Video Detection
- Flow-Guided Feature Aggregation for Video Object Detection [nips16 / iccv17] [mxnet]
- T-CNN: Tubelets with Convolution Neural Networks [cvpr16] [python]
- TPN: Tubelet Proposal Network [cvpr17] [python]
- Deep Feature Flow for Video Recognition [cvpr17] [mxnet]
- Mobile Video Object Detection with Temporally-Aware Feature Maps [cvpr18] [Google] [tensorflow]
Static Detection and Matching
Frameworks
Region Proposal
- MCG : Multiscale Combinatorial Grouping - Object Proposals and Segmentation (project) [tpami16/cvpr14] [python]
- COB : Convolutional Oriented Boundaries (project) [tpami18/eccv16] [matlab/caffe]
FPN
- Feature Pyramid Networks for Object Detection [caffe/python]
RCNN
- RFCN (author) [caffe/matlab]
- RFCN-tensorflow [tensorflow]
- PVANet: Lightweight Deep Neural Networks for Real-time Object Detection [intel] [emdnn16(nips16)]
- Mask R-CNN [tensorflow] [keras]
- Light-head R-CNN [cvpr18] [tensorflow]
- Evolving Boxes for Fast Vehicle Detection [icme18] [caffe/python]
- Cascade R-CNN (cvpr18) [detectron] [caffe]
- A MultiPath Network for Object Detection [torch] [bmvc16] [facebook]
SSD
- SSD-Tensorflow [tensorflow]
- SSD-Tensorflow (tf.estimator) [tensorflow]
- SSD-Tensorflow (tf.slim) [tensorflow]
- SSD-Keras [keras]
- SSD-Pytorch [pytorch]
- Enhanced SSD with Feature Fusion and Visual Reasoning [nca18] [tensorflow]
- RefineDet - Single-Shot Refinement Neural Network for Object Detection [cvpr18] [caffe]
YOLO
- Darknet: Convolutional Neural Networks [c/python]
- YOLO9000: Better, Faster, Stronger - Real-Time Object Detection. 9000 classes! [c/python]
- Darkflow [tensorflow]
- Pytorch Yolov2 [pytorch]
- Yolo-v3 and Yolo-v2 for Windows and Linux [c/python]
- YOLOv3 in PyTorch [pytorch]
- pytorch-yolo-v3 [pytorch] [no training] [tutorial]
- YOLOv3_TensorFlow [tensorflow]
- tensorflow-yolo-v3 [tensorflow slim]
- tensorflow-yolov3 [tensorflow slim]
- keras-yolov3 [keras]
Misc
- Relation Networks for Object Detection [cvpr18] [MXNet]
- DeNet: Scalable Real-time Object Detection with Directed Sparse Sampling [iccv17(poster)] [theano]
- SNIPER: Efficient Multi-Scale Training [cvpr18 / nips18] [mxnet]
- Multi-scale Location-aware Kernel Representation for Object Detection [cvpr18] [caffe/python]
- FCOS: Fully Convolutional One-Stage Object Detection [iccv19] [pytorch]
Matching
Boundary Detection
- Holistically-Nested Edge Detection (HED) (iccv15) [caffe]
- Edge-Detection-using-Deep-Learning (HED) [tensorflow]
- Holistically-Nested Edge Detection (HED) in OpenCV [python/c++]
- Crisp Boundary Detection Using Pointwise Mutual Information (eccv14) [matlab]
Optical Flow
- FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks (cvpr17) [caffe] [pytorch/nvidia]
- SPyNet: Spatial Pyramid Network for Optical Flow (cvpr17) [lua] [pytorch]
- Guided Optical Flow Learning (cvprw17) [caffe] [tensorflow]
- Fast Optical Flow using Dense Inverse Search (DIS) [eccv16] [C++]
- A Filter Formulation for Computing Real Time Optical Flow [ral16] [c++/cuda - matlab,python wrappers]
- PatchBatch - a Batch Augmented Loss for Optical Flow [cvpr16] [python/theano]
- Piecewise Rigid Scene Flow [iccv13/eccv14/ijcv15] [c++/matlab]
- DeepFlow v2 [iccv13] [c++/python/matlab], [project]
- An Evaluation of Data Costs for Optical Flow [gcpr13] [matlab]
Instance Segmentation
- Fully Convolutional Instance-aware Semantic Segmentation [cvpr17] [coco16 winner] [mxnet]
- Instance-aware Semantic Segmentation via Multi-task Network Cascades [cvpr16] [caffe] [coco15 winner]
- DeepMask/SharpMask [nips15/eccv16] [facebook] [torch] [tensorflow] [pytorch/deepmask]
- Simultaneous Detection and Segmentation [eccv14] [matlab] [project]
- RetinaMask [arxviv1901] [pytorch]
- Mask Scoring R-CNN [cvpr19] [pytorch]
Frameworks
- Fast, modular reference implementation of Instance Segmentation and Object Detection algorithms in PyTorch [pytorch] [facebook]
Semantic Segmentation
- Learning from Synthetic Data: Addressing Domain Shift for Semantic Segmentation [cvpr18] [spotlight] [pytorch]
- Few-shot Segmentation Propagation with Guided Networks [ax1806] [pytorch] [incomplete]
- Pytorch-segmentation-toolbox [DeeplabV3 and PSPNet] [pytorch]
- DeepLab [tensorflow]
- Auto-DeepLab [pytorch]
- DeepLab v3+ [pytorch]
Video Segmentation
- Improving Semantic Segmentation via Video Prediction and Label Relaxation [cvpr19] [pytorch] [nvidia]
Autoencoders
- β-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework [iclr17] [deepmind] [tensorflow] [tensorflow] [pytorch]
- Disentangling by Factorising [ax1806] [pytorch]
Classification
- Learning Efficient Convolutional Networks Through Network Slimming [iccv17] [pytorch]
- Deep RL
- Asynchronous Methods for Deep Reinforcement Learning
Misc
Collections
Datasets
- Awesome Public Datasets
- List of traffic surveillance datasets
- Machine learning datasets: A list of the biggest machine learning datasets from across the web
- Labeled Information Library of Alexandria: Biology and Conservation [other conservation data sets]
- THOTH: Data Sets & Images
- Google AI Datasets
- Google Cloud Storage public datasets
- Microsoft Research Open Data
- Earth Engine Data Catalog
- Registry of Open Data on AWS
- Kaggle Datasets
Single Object Tracking
- List of deep learning based tracking papers
- List of single object trackers with results on OTB
- Collection of Correlation Filter based trackers with links to papers, codes, etc
- VOT2018 Trackers repository
- CUHK Datasets
Multi Object Tracking
- List of multi object tracking papers
- A collection of Multiple Object Tracking (MOT) papers in recent years, with notes
Segmentation
- Segmentation Papers and Code
- Segmentation.X : Papers and Benchmarks about semantic segmentation, instance segmentation, panoptic segmentation and video segmentation
Deep Compressed Sensing
Misc
- Papers With Code : the latest in machine learning
- Awesome Deep Ecology
- List of Matlab frameworks, libraries and software
- Face Recognition
- A Month of Machine Learning Paper Summaries
Tutorials
Static Detection
- Deep Learning for Object Detection: A Comprehensive Review
- Review of Deep Learning Algorithms for Object Detection
- A Simple Guide to the Versions of the Inception Network
- R-CNN, Fast R-CNN, Faster R-CNN, YOLO - Object Detection Algorithms
- A gentle guide to deep learning object detection
- The intuition behind RetinaNet
- YOLO—You only look once, real time object detection explained
- Understanding Feature Pyramid Networks for object detection (FPN)
- Fast object detection with SqueezeDet on Keras
- Region of interest pooling explained
Video Detection
Instance Segmentation
- Splash of Color: Instance Segmentation with Mask R-CNN and TensorFlow
- Simple Understanding of Mask RCNN
- Learning to Segment
- Analyzing The Papers Behind Facebook's Computer Vision Approach
- Review: MNC — Multi-task Network Cascade, Winner in 2015 COCO Segmentation
- Review: FCIS — Winner in 2016 COCO Segmentation
- Review: InstanceFCN — Instance-Sensitive Score Maps