google-research/retvec

Pretrain large BERT model with RetVec - code and colab tutorial

Opened this issue · 2 comments

MarinaZhang commented
Pretrain large BERT model with RetVec - code and colab tutorial
stefan-it commented

Hi @MarinaZhang ,

many thanks for releasing the code for the RetVec paper! I would be interesting in pretraining a BERT model with RetVec so I just wanted to ask, if you e.g. have already a demo code and colab tutorial available 🤔

Many thanks!

MarinaZhang commented

Hello! are you interested in pre-training a BERT model on your own dataset or just finetuning a pre-trained one? We don't have a demo colab available yet but hopefully soon, I'll reopen this issue to work on it

  • 🚀2