jackroos/VL-BERT

how much computing resources do you spend to pretrain the model?

junchen14 opened this issue · 2 comments

junchen14 commented

Hi, recently I prepare to pretrain a similar model, I am wondering how much computing resources (the number of gpus, the type of gpu and amount of time). Hope you could answer this question to me.

jackroos commented

@junchen14 It takes 2 days for VL-BERT_base and 5.5 days for VL-BERT_large on 16 Tesla V100 (16G) GPUs to conduct the pre-training. By the way, if you freeze the Fast RCNN, the speed would be much faster, it only takes about 30 hours on 4 Tesla V100 (16G) GPUs for VL-BERT_base.

  • 👍1
junchen14 commented

thanks for your info. I appreciate it very much.