本项目属于CleanTransformer的衍生项目，包括amp、data parallel、tensor parallel、pipeline parallel、ZeRO等的复现代码
欢迎大家来一起完善代码和教程
文字教程见：
- amp
- data parallel
- pipeline parallel
- tensor parallel
- ZeRO
- Activition Checkpointing
- Model Quantization
firechecking/CleanParallel
an implementation of parallel skills like amp, ddp, pp, tp for learning purposes
Python