Enriching Local and Global Contexts for Temporal Action Localization

This repo holds the codes and models for the ContextLoc framework presented on ICCV 2021

Enriching Local and Global Contexts for Temporal Action Localization Zixin Zhu, Wei Tang, Le Wang*, Nanning Zheng, Gang Hua, ICCV 2021, Montreal, Canada.

[Paper]

Usage Guide

Usage Guide

Prerequisites

The training and testing in ContextLoc based on PGCN is reimplemented in PyTorch for the ease of use.

PyTorch 1.0.1

Other minor Python modules can be installed by running

pip install -r requirements.txt

Code and Data Preparation

Get the code

Clone this repo with git, please remember to use --recursive

git clone --recursive https://github.com/buxiangzhiren/ContextLoc.git

Download Datasets

We support experimenting with THUMOS14 datasets for temporal action detection.

THUMOS14: We need the validation videos for training and testing videos for testing. You can download them from the THUMOS14 challenge website.

Download Features

Here, we use the I3D features (RGB+Flow) of PGCN for training and testing.

THUMOS14: You can download it from Google Cloud or Baidu Cloud.

Training ContextLoc

Plesse first set the path of features in data/dataset_cfg.yaml

train_ft_path: $PATH_OF_TRAINING_FEATURES
test_ft_path: $PATH_OF_TESTING_FEATURES

Then, you can use the following commands to train ContextLoc

python ContextLoc_train.py thumos14 --snapshot_pre $PATH_TO_SAVE_MODEL

The training time of ContextLoc is long, please be patient to its completion.

After training, there will be a checkpoint file whose name contains the information about dataset and the number of epoch. This checkpoint file contains the trained model weights and can be used for testing.

Testing Trained Models

You can obtain the detection scores by running

sh test.sh TRAINING_CHECKPOINT

Here, TRAINING_CHECKPOINT denotes for the trained model. This script will report the detection performance in terms of mean average precision at different IoU thresholds.

Please first change the "result" file of RGB to the "RGB_result" file and change the "result" file of Flow to the ''Flow_result" file.

Then put them in the "results" folder.

You can obtain the two-stream results on THUMOS14 by running

sh test_two_stream.sh

Citation

Please cite the following paper if you feel ContextLoc useful to your research

@inproceedings{Zhu2021EnrichingLA,
  author={Zixin Zhu and Wei Tang and Le Wang and Nanning Zheng and G. Hua},
  title={Enriching Local and Global Contexts for Temporal Action Localization},
  booktitle   = {ICCV},
  year      = {2021},
}

Contact

For any question, please file an issue or contact

Zixin Zhu: zhuzixin@stu.xjtu.edu.cn

Reference Software Projects:

https://github.com/Alvin-Zeng/PGCN

buxiangzhiren/ContextLoc