juntang-zhuang/Adabelief-Optimizer

Question about SGD optimizer in LSTM experiments

yunfei-teng opened this issue · 1 comments

yunfei-teng commented

Hi Juntang,

Nice work indeed! The codes are quite well-written! May I ask two questions regarding SGD optimizer in LSTM experiments please?

(1) In the experiments, is there any specific reason to switch SGD optimizer to ASGD optimizer? I did not catch any related information in your paper about that.

(2) Should you use the validation dataset instead of test dataset when deciding if to switch to ASGD?

Thanks for your precious time.

Best,

juntang-zhuang commented

Hi, thanks for the question. Actually I have no idea what is ASGD, the code is just forked from https://github.com/salesforce/awd-lstm-lm/blob/master/main.py Perhaps you can ask the question there.

  • 👍1
  • 😄1