Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.
Primary LanguagePythonApache License 2.0Apache-2.0
No issues in this repository yet.