/Linear_VideoTransformer

This repo is the official implementations of "Linear Video Transformer with Feature Fixation".

Primary LanguagePython

This repo is the official implementations of "Linear Video Transformer with Feature Fixation".

Installation

Install pytorch

conda create -n video_transformer python=3.8

conda activate video_transformer

bash install.sh

Data pre-process

1.download K400, K600 and SSv2 dataset

2.extract video frames

bash data/extract_frames.sh $PATH_TO_YOUR_DATA_ROOT $PATH_TO_YOUR_SAVR_DIR

3.save the extraced frames in h5 format

python data/process.py $PATH_TO_YOUR_EXTRACTED_FRAMES $PATH_TO_YOUR_H5_SAVR_DIR

Train: change config and DataPath in the shell.

K400:

sh train_k400_linear.sh

K600:

sh train_k600_linear.sh

SSv2:

sh train_ssv2_linear.sh

Charades:

sh train_charades_linear.sh

Citation

If you use this code, please consider cite:

@article{lu2022linear,
  title={Linear Video Transformer with Feature Fixation},
  author={Lu, Kaiyue and Liu, Zexiang and Wang, Jianyuan and Sun, Weixuan and Qin, Zhen and Li, Dong and Shen, Xuyang and Deng, Hui and Han, Xiaodong and Dai, Yuchao and others},
  journal={arXiv preprint arXiv:2210.08164},
  year={2022}
}