yitu-opensource/T2T-ViT

small question about lr_scheduler

huixiancheng opened this issue · 3 comments

Thanks for the opensource code!!
Could you tell me the meaning of metric in lr step?

T2T-ViT/main.py

Lines 577 to 579 in f436fe4

if lr_scheduler is not None:
# step LR for next epoch
lr_scheduler.step(epoch + 1, eval_metrics[eval_metric])

T2T-ViT/main.py

Lines 688 to 689 in f436fe4

if lr_scheduler is not None:
lr_scheduler.step_update(num_updates=num_updates, metric=losses_m.avg)

In my understanding.Look like in timm it's don't have special meaning.

Hi, lr_scheduler is the method to set up lr decay in training, such as multi-step decay and cosine decay. We use cosine decay in this work.

I know it's cosine_schedule with warm up.
But the ques is about update the ls after a epoch or batch(iter), there are two paras: epoch/num_updates and metric.
Since I don't use timm before,I cann't understand the meaning of Para:metric
In timm it don't have special meaning too. As the code show.
QQ截图20210325112532

You need update ls in each epoch to tell it current epoch, which is the ls usage in torch.