EagleW/Stage-wise-Fine-tuning

GPU memory issue

Closed this issue · 4 comments

shixiao9941 commented

Hi Qingyun, it is really a nice job. Thank you so much for providing the codes and models. When I was try to decode your model, even with batch size 1, my machine suffers from the "CUDA out of memory" problem. I am not sure whether it is because of me or it is because of my machine. I am using V100 GPU with 32GB memory. May I ask what the GPU memory of your machine is when you finetuning and decoding the T5-large model? Hope to hear from you. Thank you so much!

EagleW commented

Hi @shixiao9941 , thank you very much for your interest in our research. Could you share more details and the training bash script you use? Do you use the recommend transformer version? Thank you!

shixiao9941 commented
EagleW commented

Great! Have a good day!

shixiao9941 commented