[Question] The optimizer used for training

[fahos]

Thank you for providing the source code.
Is the optimizer used to train the SimCSE model published on the huggingface hub AdamW?
Could you please provide more information about the optimizer used during training?
Thank you.

[gaotianyu1350]

Yes it is AdamW.

[fahos]

Thank you for your quick response.
Are each parameter of AdamW specified besides the learning rate the default values for transformers' AdamW?
https://huggingface.co/docs/transformers/v4.46.0/en/main_classes/optimizer_schedules#transformers.AdamW

[gaotianyu1350]

Hi, yes, and you should be able to reproduce our results with the scripts provided in this repo.

[fahos]

Thank you for answering my question. I really appreciate it.