Official code for "SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient"
Primary LanguagePython