About learning-rate when training with one GPU
Closed this issue · 1 comments
dzgx commented
Thank you for your work.
I have found that your training with 4GPUS where you set the learning-rate 0.01 which you have referred it in the README.md
Is there any skill or relation of setting lr about GPU?
cuiziteng commented
Yeah, assume same batch size per GPU, generally the learning rate is linear growth with your GPUs number, like 0.01 for 4GPUs, and 0.005 for 2GPUs, 0.02 for 8GPUs.