Training:
python run_downsteam -m train -u <upstream model name> -d <downstream task name> -s <feature type>
TBA
TBA
@article{tseng2023avsuperb,
title={AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models},
author={Yuan Tseng and Layne Berry and Yi-Ting Chen and I-Hsiang Chiu and Hsuan-Hao Lin and Max Liu and Puyuan Peng and Yi-Jen Shih and Hung-Yu Wang and Haibin Wu and Po-Yao Huang and Chun-Mao Lai and Shang-Wen Li and David Harwath and Yu Tsao and Shinji Watanabe and Abdelrahman Mohamed and Chi-Luen Feng and Hung-yi Lee},
journal={arXiv preprint arXiv:2309.10787},
year={2023}
}
AV-SUPERB is primarily distributed under the terms of both the MIT license and the Apache License (Version 2.0).
Using files and pretrained AV-HuBERT models under the upstream_models/vhubert
folder requires accepting the terms in the AV-HuBERT license agreement listed in this file.
See LICENSE-APACHE, LICENSE-MIT, COPYRIGHT for details.
Source code is based on S3PRL.