OptimalScale/LMFlow

How can I use other optimizers in LISA?

Opened this issue · 1 comments

I tried to pass --optim , nothing happens. How can I use optimizers such as adamw_8bit or adafactor in LISA?
They are not in the custom_optimizers either.

Thanks for your interest in LMFlow! Normally LISA should effect all optimizers. We are wondering if you could provide the screenshot of the problem so we could check that for you? Thanks very much for providing the suggestion 😄