UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()’

Question

UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()’

vishnuvskvkl opened this issue 3 years ago · 2 comments

Answer 1 · 2022-07-13T02:31:14.000Z

How can I resolve this issue? can i simply put optimizer.step() before scheduler.step()

Answer 2 · 2022-07-13T05:22:49.000Z

If you install the pytorch==1.1, the above warning will disappear.

However, this warning is caused from the warm-up learning rate strategy for the first 3 epochs.

According to the training log and our experiments, the problem doesn't affect the training process.

Or you can remove the warm-up strategy, only apply the cosine annealing by changing these lines for the first 3 epochs) to scheduler = optim.lr_scheduler.CosineAnnealingLR(optimizer, OPT['EPOCHS'], eta_min=float(OPT['LR_MIN'])).