Question about reproducibility
daisukelab opened this issue · 2 comments
Hello, thanks for your effort to make it easier to understand the data2vec.
Let me ask a quick question; can we reproduce the paper with your implementation?
I guess it is out of the scope of this repo, but I thought it would be quite nice if possible.
Thank you anyway!
Hello Daisuke, I'm so glad this repo has been useful!
Actually, I attempted to copy the weights from HuggingFace version (which is copied from fairseq itself!) into the models in this repo. Although I highly recommend you use the models in HuggingFace, there still is the option to load the exact same weights as the original paper. You can find out more in the README.md.
Thanks for your comment. I understood that pretraining can be done, but has not been done for reproducing paper; we are to use weights from Hugging Face.
Thanks again! :)