yongliang-wu/Repurpose

Repurpose from User Generated Content Videos: A Large-scale Dataset and Benchmark

Summary

• Introduction
• Getting Started
• Datasets

Introduction

Getting Started

Setting Up Your Environment

To ensure a smooth experience running the scripts, set up a dedicated conda environment by executing the following commands in your terminal:

conda create -n repurpose python=3.9
conda activate repurpose
pip install -r requirements.txt

Preparing Your Data

The train/validation/test splits are provided in the /data directory. Follow these steps for data preparation:

1. Download the source videos using yt-dlp.
2. Extract the required features as mentioned in our paper using these repositories:
   • video_features
   • panns_inference
   • sentence-transformers
   • whisperX

Training Your Model

To begin training the model, use the command below:

python main.py --config_path configs/Repurpose.yaml

For model evaluation, execute the following command:

python inference.py --config_path configs/Repurpose.yaml --resume your_ckpt_path

Replace your_ckpt_path with the actual path to your checkpoint file.

Acknowledgments

We would like to extend our gratitude to the authors and contributors of the following repositories, which have been instrumental in the development of our implementation:

• UnAV
• Soft-NMS