How to start deepspeed automatically?

[qwerfdsadad]

If there is a main.py, I want to train with deepspeed through a parameter, since the commands for these two training methods are different.
Such as, if the parameter==true, training with deepspeed, else training in the normal way.
Further, how to automatically choose to train with zero 1 or 2 or 3?

[loadams]

Hi @qwerfdsadad - you should be able to start with the docs here to see how to conditionally wrap your model with DeepSpeed.

Zero stages 1,2, and 3 come with different tradeoffs, you do not need to use them, but they are increasingly useful if you run out of memory. Docs here

[loadams]

Hi @qwerfdsadad - closing this as stale for now. Please comment if you need more assistance.