--restore_training doesnt restore learning rate

[cameronbergh]

the --restore_training arg doesnt restore the learning rate. i suppose the remedy would be to set it manually with the --lr arg?

[wangcongcong123]

@cameronbergh to answer your question on restore training, if the lr scheduler is constant then I think it is straightforward to keep lr the same as last training (so just manually set it up). If it is dynamic lr-based scheduler like linear warmup, this is a bit tricky and so far the code has not saved the lr from previous training. Anyway, this can be done in the next iteration