Pretrain large BERT model with RetVec - code and colab tutorial
Opened this issue · 2 comments
MarinaZhang commented
Pretrain large BERT model with RetVec - code and colab tutorial
stefan-it commented
Hi @MarinaZhang ,
many thanks for releasing the code for the RetVec paper! I would be interesting in pretraining a BERT model with RetVec so I just wanted to ask, if you e.g. have already a demo code and colab tutorial available 🤔
Many thanks!
MarinaZhang commented
Hello! are you interested in pre-training a BERT model on your own dataset or just finetuning a pre-trained one? We don't have a demo colab available yet but hopefully soon, I'll reopen this issue to work on it