Reproduction of machine translation results

Question

Reproduction of machine translation results

Opened this issue 2 years ago · 2 comments

Hi~
Is it possible to release SAGE's code for machine translation tasks?

Answer 1 · 2022-11-24T22:18:50.000Z

Hi @zwhe99 , I have no upcoming plan of releasing it - I no longer have access to the server that stores the code, unfortunately.

However, it should be easy to implement yourself. Our implementation was based on fairseq. You might add an AdamW-SAGE class in https://github.com/facebookresearch/fairseq/tree/main/fairseq/optim following UnstructAwareAdamW in this repo.

Answer 2 · 2022-12-01T02:30:13.000Z

Hi @cliang1453, I found that you define different param groups with different 'params_type' and 'weight_decay' here. Did you do the same in the fairseq version?