A curated list of temporal action localization/detection and related area (e.g. temporal action proposal) resources.
- [MUSeS] Multi-shot temporal event localization: A Benchmark - Xiaolong Liu et al,
CVPR 2021
- [SALAD] SALAD: Self-Assessment Learning for Action Detection - Guillaume Vaudaux-Ruth et al,
WACV 2021
- [RTD-Net] Relaxed Transformer Decoders for Direct Action Proposal Generation - Jing Tan et al,
arxiv 2021
. [code] - [AGT] Activity Graph Transformer for Temporal Action Localization - Megha Nawhal et al,
arxiv 2021
- [VSGN] Video Self-Stitching Graph Network for Temporal Action Localization - Chen Zhao et al,
arxiv 2020
- [UFA] Temporal Action Detection with Multi-level Supervision - Baifeng Shi et al,
arxiv 2020
- [TSP] TSP: Temporally-Sensitive Pretraining of Video Encoders for Localization Tasks - Humam Alwassel et al,
arxiv 2020
- [BSP] Boundary-sensitive Pre-training for Temporal Localization in Videos - Mengmeng Xu et al,
arxiv 2020
- [VAN] Temporal Action Localization with Variance-Aware Networks - Ting-Ting Xie et al,
arxiv 2020
- [TSI] TSI: Temporal Scale Invariant Network for Action Proposal Generation - Shuming Liu et al,
ACCV 2020
. [code] - [BU-TAL] Bottom-Up Temporal Action Localization with Mutual Regularization - Peisen Zhao et al,
ECCV 2020
. - [DBG] Fast Learning of Temporal Action Proposal via Dense Boundary Generator - Chunming Lin et al,
AAAI 2020
. [code] - [G-TAD] G-TAD: Sub-Graph Localization for Temporal Action Detection - Mengmeng Xu et al,
CVPR 2020
. [code] - [PBRNet] Progressive Boundary Refinement Network for Temporal Action Detection - Qinying Liu et al,
AAAI 2020
. - [AGCN] Graph Attention based Proposal 3D ConvNets for Action Detection - Jun Li et al,
AAAI 2020
.
- [PGCN] Graph Convolutional Networks for Temporal Action Localization - Runhao Zeng et al,
ICCV 2019
. [code] - [RAM] Relation Attention for Temporal Action Localization - Peihao Chen et al,
TMM 2019
. - [BMN] BMN: Boundary-Matching Network for Temporal Action Proposal Generation - Tianwei Lin et al,
ICCV 2019
. - [GTAN] Gaussian Temporal Awareness Networks for Action Localization - Fuchen Long et al,
CVPR 2019
. - [DBS] Video Imprint Segmentation for Temporal Action Detection in Untrimmed Videos - Zhanning Gao et al,
AAAI 2019
. - [C-TCN] Deep Concept-wise Temporal Convolutional Networks for Action Localization - Xin Li et al,
arXiv 2019
.
- [TAL-Net] Rethinking the Faster R-CNN Architecture for Temporal Action Localization - Yuwei Chao et al,
CVPR 2018
. - [BSN] BSN: Boundary Sensitive Network for Temporal Action Proposal Generation - Tianwei Lin et al,
ECCV 2018
. [code] - [Action-Search] Action Search: Spotting Actions in Videos and Its Application to Temporal Action Localization - Humam Alwassel et al,
ECCV 2018
. [code] - [TPC] Exploring Temporal Preservation Networks for Precise Temporal Action Localization - Ke Yang et al,
AAAI 2018
. - [Self-Ad] A Self-Adaptive Proposal Model for Temporal Action Detection based on Reinforcement Learning - Jingjia Huang et al,
AAAI 2018
.
- [SSN] Temporal Action Detection with Structured Segment Networks - Yue Zhao et al,
ICCV 2017
. [code] - [R-C3D] R-C3D: Region Convolutional 3D Network for Temporal Activity Detection - Huijuan Xu et al,
ICCV 2017
. [code] - [TCN] Temporal Context Network for Activity Localization in Videos - Xiyang Dai et al,
ICCV 2017
. - [TURN] TURN TAP: Temporal Unit Regression Network for Temporal Action Proposals - Jiyang Gao et al,
ICCV 2017
. [code] - [SST] SST: Single-Stream Temporal Action Proposals - Shyamal Buch et al,
ICCV 2017
. - [CDC] CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videos - Zheng Shou et al,
CVPR 2017
. [code] - [SCC] SCC: Semantic Context Cascade for Efficient Action Detection - Fabian Caba Heilbron et al,
CVPR 2017
. - [SMS] Temporal Action Localization by Structured Maximal Sums - Zehuan Yuan et al,
CVPR 2017
.
- [S-CNN] Temporal Action Localization in Untrimmed Videos via Multi-stage CNNs - Zheng Shou et al,
CVPR 2016
. [code] - [PSDF] Temporal Action Localization with Pyramid of Score Distribution Features - Jun Yuan et al,
CVPR 2016
. - [FG] End-to-end Learning of Action Detection from Frame Glimpses in Videos - Serena Yeung et al,
CVPR 2016
. - [SLM] Temporal Action Detection Using a Statistical Language Model - Alexander Richard et al,
CVPR 2016
. - [DAPs] DAPs: Deep Action Proposals for Action Understanding - Victor Escorcia et al,
ECCV 2016
.
Method | Conference | IoU=0.1 | IoU=0.2 | IoU=0.3 | IoU=0.4 | IoU=0.5 | IoU=0.6 | IoU=0.7 |
---|---|---|---|---|---|---|---|---|
DAPs | ECCV-2016 | - | - | - | - | 13.9 | - | - |
SLM | CVPR-2016 | 39.7 | 35.7 | 30.0 | 23.2 | 15.2 | - | - |
FG | CVPR-2016 | 48.9 | 44.0 | 36.0 | 26.4 | 17.1 | - | - |
SMS | CVPR-2017 | 51.0 | 45.2 | 36.5 | 27.8 | 17.8 | - | - |
PSDF | CVPR-2016 | 51.4 | 42.6 | 33.6 | 26.1 | 18.8 | - | - |
S-CNN | CVPR-2016 | 47.7 | 43.5 | 36.3 | 28.7 | 19.0 | 10.3 | 5.3 |
SST | ICCV-2017 | - | - | - | - | 23.0 | - | - |
CDC | CVPR-2017 | - | - | 40.1 | 29.4 | 23.3 | 13.1 | 7.9 |
TURN | ICCV-2017 | 54.0 | 50.9 | 44.1 | 34.9 | 25.6 | - | - |
TCN | ICCV-2017 | - | - | - | 33.3 | 25.6 | 15.9 | 9.0 |
Self-Ad | AAAI-2018 | - | - | - | - | 27.7 | - | - |
TPC | AAAI-2018 | - | - | 44.1 | 37.1 | 28.2 | 20.6 | 12.7 |
R-C3D | ICCV-2017 | 54.5 | 51.5 | 44.8 | 35.6 | 28.9 | - | - |
SSN | ICCV-2017 | 66.0 | 59.4 | 51.9 | 41.0 | 29.8 | - | - |
Action-Search | ECCV-2018 | - | - | 51.8 | 42.4 | 30.8 | 20.2 | 11.1 |
DBS | AAAI-2019 | 56.7 | 54.7 | 50.6 | 43.1 | 34.3 | 24.4 | 14.7 |
BSN | ECCV-2018 | - | - | 53.5 | 45.0 | 36.9 | 28.4 | 20.0 |
AGCN | AAAI-2020 | 59.3 | 59.6 | 57.1 | 51.6 | 38.6 | 28.9 | 17.0 |
GTAN | CVPR-2019 | 69.1 | 63.7 | 57.8 | 47.2 | 38.8 | - | - |
BMN | ICCV-2019 | - | - | 56.0 | 47.4 | 38.8 | 29.7 | 20.5 |
DBG | AAAI-2020 | - | - | 57.8 | 49.4 | 39.8 | 30.2 | 21.7 |
TAL-Net | CVPR-2018 | 59.8 | 57.1 | 53.2 | 48.5 | 42.8 | 33.8 | 20.8 |
RAM | TMM-2019 | 65.4 | 63.1 | 58.8 | 52.7 | 43.7 | - | - |
PGCN | ICCV-2019 | 69.5 | 67.8 | 63.6 | 57.8 | 49.1 | - | - |
PBRNet | AAAI-2020 | - | - | 58.5 | 54.6 | 51.3 | 41.8 | 29.5 |
G-TAD | CVPR-2020 | - | - | 66.4 | 60.4 | 51.6 | 37.6 | 22.9 |
BU-TAL | ECCV-2020 | - | - | 53.9 | 50.7 | 45.4 | 38.0 | 28.5 |
TSI | ACCV-2020 | - | - | 61.0 | 52.1 | 42.6 | 33.2 | 22.4 |
SALAD | WACV-2021 | 73.3 | 70.7 | 65.7 | 57.0 | 44.6 | - | - |
MUSES | CVPR-2021 | - | - | 68.9 | 64.0 | 56.9 | 46.3 | 31.0 |
Method | Conference | IoU=0.1 | IoU=0.2 | IoU=0.3 | IoU=0.4 | IoU=0.5 | IoU=0.6 | IoU=0.7 |
---|---|---|---|---|---|---|---|---|
C-TCN | arXiv | 72.2 | 71.4 | 68.0 | 62.3 | 52.1 | - | - |
VSGN | arXiv | - | - | 66.7 | 60.4 | 52.4 | 41.0 | 30.4 |
UFA | arXiv | - | - | 45.6 | 36.4 | 26.2 | 15.5 | 7.1 |
TSP | arXiv | - | - | 69.1 | 63.3 | 53.5 | 40.4 | 26.0 |
VAN | arXiv | - | - | 55.0 | 48.6 | 39.2 | 26.9 | 15.0 |
AGT | arXiv | 72.1 | 69.8 | 65.0 | 58.1 | 50.2 | - | - |
RTD-Net | arXiv | - | - | 68.3 | 62.3 | 51.9 | 38.8 | 23.7 |
Method | Conference | IoU=0.5 | IoU=0.75 | IoU=0.95 | Avg |
---|---|---|---|---|---|
R-C3D | ICCV-2017 | 26.8 | - | - | - |
AGCN | AAAI-2020 | 30.4 | - | - | - |
SCC | CVPR-2017 | 39.9 | 18.7 | 4.7 | 19.3 |
TAL-Net | CVPR-2018 | 38.23 | 18.30 | 1.30 | 20.22 |
RAM | TMM-2019 | 36.99 | 23.10 | 3.34 | 23.03 |
TCN | ICCV-2017 | 37.49 | 23.47 | 4.47 | 23.58 |
CDC | CVPR-2017 | 45.3 | 26.0 | 0.2 | 23.8 |
DBS | CVPR-2019 | 43.2 | 25.8 | 6.1 | 26.1 |
PGCN | ICCV-2019 | 42.90 | 28.14 | 2.47 | 26.99 |
SSN | ICCV-2017 | 43.26 | 28.70 | 5.63 | 28.28 |
BSN | ECCV-2018 | 46.45 | 29.96 | 8.02 | 30.03 |
BMN | ICCV-2019 | 50.07 | 34.78 | 8.29 | 33.85 |
G-TAD | CVPR-2020 | 50.36 | 34.60 | 9.02 | 34.09 |
GTAN | CVPR-2019 | 52.61 | 34.14 | 8.91 | 34.31 |
PBRNet | AAAI-2020 | 53.96 | 34.97 | 8.98 | 35.01 |
BU-TAL | ECCV-2020 | 43.47 | 33.91 | 9.21 | 30.12 |
TSI | ACCV-2020 | 51.18 | 35.02 | 6.59 | 34.15 |
SALAD | WACV-2021 | 51.72 | 31.21 | 3.33 | 31.02 |
MUSES | CVPR-2021 | 50.02 | 34.97 | 6.57 | 33.99 |
Method | Conference | IoU=0.5 | IoU=0.75 | IoU=0.95 | IoU=Avg |
---|---|---|---|---|---|
C-TCN | arXiv | 47.6 | 31.9 | 6.2 | 31.1 |
VSGN | arXiv | 52.4 | 36.0 | 8.4 | 35.1 |
TSP | arXiv | 51.3 | 37.2 | 9.3 | 35.8 |
BSP | arXiv | 50.1 | 34.7 | 7.9 | 34.0 |
RTD-Net | arXiv | 46.4 | 30.4 | 8.6 | 30.5 |
- [HAM-Net] A Hybrid Attention Mechanism for Weakly-Supervised Temporal Action Localization - Ashraful Islam et al,
arxiv 2021
. [code]
- [ECM] Equivalent Classification Mapping for Weakly Supervised Temporal Action Localization - Le Yang et al,
arxiv 2020
- [TCA] Learning Temporal Co-Attention Models for Unsupervised Video Action Localization - Guoqiang Gong et al,
CVPR 2020
- [EM-MIL] Weakly-Supervised Action Localization with Expectation-Maximization Multi-Instance - Zhekun Luo et al,
ECCV 2020
. - [SF-Net] SF-Net: Single-Frame Supervision for Temporal Action Localization - Fan Ma et al,
ECCV 2020
. [code] - [A2CL-PT] Adversarial Background-Aware Loss for Weakly-supervised Temporal Activity Localization - Kyle Min et al,
ECCV 2020
. - [TSCN] Two-Stream Consensus Network for Weakly-Supervised Temporal Action Localization - Yuanhao Zhai et al,
ECCV 2020
. - [ActionBytes] ActionBytes: Learning from Trimmed Videos to Localize Actions - Mihir Jain et al,
CVPR 2020
. - [DGAM] Weakly-Supervised Action Localization by Generative Attention Modeling - Baifeng Shi et al,
CVPR 2020
. - [RPN] Relational Prototypical Network for Weakly Supervised Temporal Action Localization - Linjiang Huang et al,
AAAI 2020
. - [BaSNet] Background Suppression Network for Weakly-supervised Temporal Action Localization - Pilhyeon Lee et al,
AAAI 2020
. - [DML] Weakly Supervised Temporal Action Localization Using Deep Metric Learning - Ashraful Islam et al,
WACV 2020
. - [MCASL] Action Graphs: Weakly-supervised Action Localization with Graph Convolution Networks - Maheen Rashid et al,
WACV 2020
. - [WSGN] Weakly Supervised Gaussian Networks for Action Detection - Basura Fernando et al,
WACV 2020
.
- [MAAN] Marginalized Average Attentional Network for Weakly Supervised Learning - Yuan Yuan et al,
ICLR 2019
. - [IWO-Net] Breaking Winner-Takes-All: Iterative-Winners-Out Networks for Weakly Supervised Temporal Action Localization - Runhao Zeng et al,
TIP 2019
. - [3C-Net] 3C-Net: Category Count and Center Loss for Weakly-Supervised Action Localization - Sanath Narayan et al,
TIP 2019
. [code] - [BM] Weakly-supervised Action Localization with Background Modeling - Phuc Xuan Nguyen et al,
ICCV 2019
. - [TSM] Temporal Structure Mining for Weakly Supervised Action Detection - Tan Yu et al,
ICCV 2019
. - [CleanNet] Weakly Supervised Temporal Action Localization through Contrast based Evaluation Networks - Ziyi Liu et al,
ICCV 2019
. - [CMCS] Completeness Modeling and Context Separation for Weakly Supervised Temporal Action Localization - Daochang Liu et al,
CVPR 2019
. - [STAR] Segregated Temporal Assembly Recurrent Networks for Weakly Supervised Multiple Action Detection - Yunlu Xu et al,
AAAI 2019
.
- [W-TALC] W-TALC: Weakly-supervised Temporal Activity Localization and Classification - Sujoy Paul et al,
ECCV 2018
. [code] - [AutoLoc] AutoLoc: Weakly-supervised Temporal Action Localization in Untrimmed Videos - Zheng Shou et al,
ECCV 2018
. [code] - [STPN] Weakly Supervised Action Localization by Sparse Temporal Pooling Network - Phuc Nguyen et al,
CVPR 2018
. - [One-Shot] One-Shot Action Localization by Learning Sequence Matching Network - Hongtao Yang et al,
CVPR 2018
.
- [UNet] UntrimmedNets for Weakly Supervised Action Recognition and Detection - Limin Wang et al,
CVPR 2017
. [code] - [H&S] Hide-and-Seek: Forcing a Network to be Meticulous for Weakly-supervised Object and Action Localization - Krishna Kumar Singh et al,
CVPR 2017
.
Method | Conference | IoU=0.1 | IoU=0.2 | IoU=0.3 | IoU=0.4 | IoU=0.5 | IoU=0.6 | IoU=0.7 |
---|---|---|---|---|---|---|---|---|
H&S | ICCV-2017 | 36.44 | 27.84 | 19.49 | 12.66 | 6.84 | - | - |
UNet | CVPR-2017 | 44.4 | 37.7 | 28.2 | 21.1 | 13.7 | - | - |
One-Shot | CVPR-2018 | - | - | - | - | 14.7 | - | - |
STPN | CVPR-2018 | 52.0 | 44.7 | 35.5 | 25.8 | 16.9 | 9.9 | 4.3 |
IWO-Net | TIP-2019 | 57.6 | 48.9 | 38.9 | 29.3 | 20.5 | - | - |
MAAN | ICLR-2019 | 59.8 | 50.8 | 41.1 | 30.6 | 20.3 | 12.0 | 6.9 |
WSGN | WACV-2020 | 55.3 | 47.6 | 38.9 | 30.0 | 21.1 | - | - |
AutoLoc | ECCV-2018 | - | - | 35.8 | 29.0 | 21.2 | 13.4 | 5.8 |
W-TAL | ECCV-2018 | 55.2 | 49.6 | 40.1 | 31.1 | 22.8 | - | 7.6 |
STAR | AAAI-2019 | 68.8 | 60.0 | 48.7 | 34.7 | 23.0 | - | - |
CMCS | CVPR-2019 | 57.4 | 50.8 | 41.2 | 32.1 | 23.1 | 15.0 | 7.0 |
CleanNet | ICCV-2019 | - | - | 37.0 | 30.9 | 23.9 | 13.9 | 7.1 |
TSM | ICCV-2019 | - | - | 39.5 | - | 24.5 | - | 7.1 |
MCASL | WACV-2020 | 63.7 | 56.9 | 47.3 | 36.4 | 26.1 | - | - |
3C-Net | ICCV-2019 | 59.1 | 53.5 | 44.2 | 34.1 | 26.6 | - | 8.1 |
BM | ICCV-2019 | 60.4 | 56.0 | 46.6 | 37.5 | 26.8 | 17.6 | 9.0 |
BaSNet | AAAI-2020 | 58.2 | 52.3 | 44.6 | 36.0 | 27.0 | 18.6 | 10.4 |
RPN | AAAI-2020 | 62.3 | 57.0 | 48.2 | 37.2 | 27.9 | 16.7 | 8.1 |
DML | AAAI-2020 | 62.3 | - | 46.8 | - | 29.6 | - | 9.7 |
DGAM | CVPR-2020 | 60.0 | 54.2 | 46.8 | 38.2 | 28.8 | 19.8 | 11.5 |
ActionBytes | CVPR-2020 | - | - | 43.0 | 35.8 | 29.0 | - | 9.5 |
A2CL-PT | ECCV-2020 | 61.2 | 56.1 | 48.1 | 39.0 | 30.1 | 19.2 | 10.6 |
SF-Net | ECCV-2020 | 71.0 | 63.4 | 53.2 | 40.7 | 29.3 | 18.4 | 9.6 |
EM-MIL | ECCV-2020 | 59.1 | 52.7 | 45.5 | 36.8 | 30.5 | 22.7 | 16.4 |
TCA | CVPR-2020 | - | - | 46.9 | 38.9 | 30.1 | 19.8 | 10.4 |
Method | Conference | IoU=0.1 | IoU=0.2 | IoU=0.3 | IoU=0.4 | IoU=0.5 | IoU=0.6 | IoU=0.7 |
---|---|---|---|---|---|---|---|---|
ECM | arXiv | 62.6 | 55.1 | 46.5 | 38.2 | 29.1 | 19.5 | 10.9 |
HAM-Net | arXiv | 65.4 | 59.0 | 50.3 | 41.1 | 31.0 | 20.7 | 11.2 |
Method | Conference | IoU=0.5 | IoU=0.75 | IoU=0.95 | IoU=Avg |
---|---|---|---|---|---|
STPN | CVPR-2018 | 29.3 | 16.9 | 2.6 | 20.07 |
IWO-Net | TIP-2019 | 29.8 | 17.6 | 4.7 | - |
TSM | ICCV-2019 | 30.3 | 19.0 | 4.5 | - |
STAR | AAAI-2019 | 31.1 | 18.8 | 4.7 | - |
CMCS | CVPR-2019 | 34.0 | 20.9 | 5.7 | 21.2 |
BaSNet | AAAI-2019 | 34.5 | 22.5 | 4.9 | 22.2 |
MAAN | ICLR-2019 | 33.7 | 21.9 | 5.5 | - |
BM | ICCV-2019 | 36.4 | 19.2 | 2.9 | - |
A2CL-PT | ECCV-2020 | 36.8 | 22.0 | 5.2 | 22.5 |
Method | Conference | IoU=0.5 | IoU=0.75 | IoU=0.95 | IoU=Avg |
---|---|---|---|---|---|
ECM | arxiv | 36.7 | 23.6 | 5.9 | 23.5 |
Method | Conference | IoU=0.5 | IoU=0.75 | IoU=0.95 | IoU=Avg |
---|---|---|---|---|---|
UNet | CVPR-2017 | 7.4 | 3.2 | 0.7 | - |
AutoLoc | ECCV-2018 | 27.3 | 15.1 | 3.3 | - |
TSM | ICCV-2019 | 28.3 | 17.0 | 3.5 | - |
MCASL | AAAI-2020 | 29.4 | - | - | - |
STAR | AAAI-2019 | 31.1 | 18.8 | 4.7 | - |
DML | AAAI-2020 | 35.2 | - | - | - |
W-TALC | ECCV-2018 | 37.0 | - | - | 18.0 |
CleanNet | ICCV-2019 | 37.1 | 20.3 | 5.0 | 21.6 |
3C-Net | ICCV-2019 | 37.2 | - | - | - |
CMCS | CVPR-2019 | 36.8 | 22.0 | 5.6 | 22.4 |
RPN | AAAI-2020 | 37.6 | 23.9 | 5.4 | 23.3 |
BaSNet | AAAI-2020 | 38.5 | 24.2 | 5.6 | 24.3 |
ActionBytes | CVPR-2020 | 39.4 | - | - | - |
EM-MIL | ECCV-2020 | 37.4 | - | - | - |
TCA | CVPR-2020 | 40.0 | 25.0 | 4.6 | 24.6 |