[AdamW] amsgrad issue

[frgfm]

Hi there,

Thanks a lot for the implementation! Out of curiosity, I tested the other optimizers apart from radam. And I think there is a missing argument in the constructor of AdamW on line 154:

def __init__(self, params, lr=1e-3, betas=(0.9, 0.999), eps=1e-8, weight_decay=0, amsgrad=False, warmup=0):

Otherwise calling the constructor throws the following error:

  File "main.py", line 125, in main
    optimizer = AdamW(model_params, weight_decay=args.weight_decay)
  File "/home/fg/fv-training/optimizer.py", line 156, in __init__
    weight_decay=weight_decay, amsgrad=amsgrad, warmup = warmup)
NameError: name 'amsgrad' is not defined

Cheers

[LiyuanLucasLiu]

Thanks for bringing this up, Nice catch!