I have some questions in your paper and code

Question

I have some questions in your paper and code

fsfwl11406 opened this issue 3 years ago · 1 comments

Which code corresponds to Stage 4 in your paper ?Anneal_mode in your hyperparams is "cosine",but your code only use mode="hard",and where you give num_warmup_steps assignment.

Answer 1 · 2021-10-13T11:02:03.000Z

About Stage 4, you can refer Step.3 in Usage part in README.md. Following the command, you can get the corresponding codes.

The anneal_mode is set as "cosine" in hyperparams.py. All of hyperparameters are set in hyperparams.py actually. Please check it. Thank you.