How to fintuneing Whisper with How2 subset for AVSR task

[Zth9730]

Thanks for your working.
As for the avsr task, I seem not to find the code that fintune whisper with the subset of how2 dataset? How can i do that?

[jasonppy]

Thanks for your question. I did not finetune Whisper on how2 (I didn't finetune whisper on any tasks for this paper). It's just the number of retrieved objects i.e. obj_topk, as a hyperparameter, was found using a subset of how2. This means we used the same script as we did for visspeech, you just need to run it on your validation set a couple of times using different obj_topk and select the best

[Zth9730]

Thank you very much for your prompt response!