How to control the times of optimizer.step() for different level?
Closed this issue · 1 comments
Hello, I'm not an expert on MLO. If I understand correctly, in one iteration, the level-2 module and level-1 module are all updated once (that is, their optimizer.step() is called both once)?
My question is how can I control this. For example, I want to have level-2 calls optimizer.step() for 100 times and then calls optimizer.step() 1 time for level-1. Thanks!
Sorry for the late response! Unfortunately, we only support larger lower-level optimizer.step(). Basically, you can control this with unroll_steps in Config. For example, if you set unroll_steps=100 for level 1, you will perform level 1 optimizer.step() 100 times before performing optimizer.step() for level 2. Sorry for the inconvenience. Let me know if you have further questions!
Best,
Sang