ydhongHIT opened this issue 4 years ago · 1 comments
The default lr rate is 5e-4 rather than 1e-3. So which lr rate should I use to reproduce your results?
Lr is linearly scaled based on batch size. Reference lr rate is 0.0005 with batch size of 512.