/VideoABC

[TIP 2022] VideoABC: A Real-World Video Dataset for Abductive Visual Reasoning

Primary LanguagePython

VideoABC

Created by Wenliang Zhao, Yongming Rao, Yansong Tang, Jie Zhou, Jiwen Lu

This repository contains PyTorch implementation for VideoABC: A Real-World Video Dataset for Abductive Visual Reasoning (TIP 2022).

[IEEE Xplore]

Dataset

VideoABC is built from the COIN dataset. Please download the videos from COIN, and place them in the data folder.

├── configs
├── data
│   ├── ...
│   ├── A129SM9S54A
│   ├── a158bYDFSwU
│   ├── A1AjNjJFGs4
│   └── ...
├── metadata
├── README.md
├── slowfast
├── tools
└── ...

The metadata folder contains the VideoABC question/choice pairs as well as the train/test split.

Training

python tools/run_net.py --cfg configs/VideoABC/SLOWFAST_HDR.yaml

Evaluation

python tools/run_net.py --cfg configs/VideoABC/SLOWFAST_HDR.yaml TEST.CHECKPOINT_FILE_PATH path/to/checkpoint TEST.ENABLE True

Acknowledgement

Our code is based on SlowFast. Our dataset is based on COIN.

Citation

If you find our work useful in your research, please consider citing:

@article{zhao2022videoabc,
  title={VideoABC: A Real-World Video Dataset for Abductive Visual Reasoning},
  author={Zhao, Wenliang and Rao, Yongming and Tang, Yansong and Zhou, Jie and Lu, Jiwen},
  journal={IEEE Transactions on Image Processing},
  volume={31},
  pages={6048--6061},
  year={2022},
  publisher={IEEE}
}