Ze-Yang/Context-Transformer

How can I reproduce experiment in limit cuda memory

Closed this issue · 3 comments

xingE650 commented

Your work is excellent, and by reading your readme.md I can easily reproduce your result in VOC increment experiment setting. However, due to my limit cuda memory(I have only two 2080ti for experiment), my result is lower than your paper(loss 5%) in batch size 32. Can you give me some suggestion on improving experiment result? Thank you very much!

Ze-Yang commented

If memory is not sufficient and you have to reduce batch size, I suggest following the linear lr scaling recipe: in the case of 2 GPUs, set batchsize = 8 (keep 16 ims/GPU unchanged), multiply the iteration-based steps and max-iter by 2x (keep # epochs unchanged), and set lr by 0.5x as 2e-3 (linear lr). This is not fully equivalent because the BN size is changed, but I think the results would be similar if the batch size is not too small.

  • 👍1
xingE650 commented

Thanks for you reply. I use the linear lr scaling recipe with batch size=32, it improve my result almost 2% mAP and the results are more similar with your result.

Ze-Yang commented

Great to hear that. I close this issue now.