GPT2 training
ivcy0928 opened this issue · 1 comments
ivcy0928 commented
It is such an excellent work!
However, I am wondering is GPT2（audiomae feature generating model）pretrained before joint training. For I cannot found training script for GPT2 module.
By the way, the example "AudioCaps Dataset" is unable to be fetched for its large size and limitations of my poor network. Is it posible if an explaination of the dataset is provided so that I can directly preprocess my local dataset.
Thanks very much!
CJ416 commented
have you found the code related to GPT2 pretraining?