About learning rate setting

[hsi-che-lin]

Hello,
In most of the config files (e.g., this), I can see that the argument "lr" of the optimizers are set to 3e-4, and warmup_iters of the lr_config is set to 2.5. Does this mean the learning rate is starting from 0, linearly increases to 3e-4 at 2.5 epoch, and anneals back to 0 at 50 epoch? Do you scale the learning rate based on the batch size?

[taoyang1122]

Hi, you understanding about the behavior of learning rate decay is correct. You may want to scale the learning rate based on the batch size. The current learning rate is for batch size 64.