kwai/Megatron-Kwai
[USENIX ATC '24] Accelerating the Training of Large Language Models using Efficient Activation Rematerialization and Optimal Hybrid Parallelism
PythonNOASSERTION
Stargazers
- aeeeeeepYCTU
- AndyBug0Shanghai AI Lab
- BearBiscuit05HeiFei
- ChenQiaoling00
- chenyuwang814University of California, Los Angeles
- constroyShanghai AI Laboratory
- demonatic
- Desperado-JiaBeijing
- duxin199508
- ETOgaosionICT
- fredbjer
- ftgreat
- Gy-Lu@hpcaitech
- gyeongchan-yun
- ht-zhou
- Infi-zc
- JiwenJInstitute of Automation, Chinese Academy of Sciences
- Kangkang-wkyBeijing
- kssamwangUniversity of Science and Technology of China
- leleucas
- liaoyiqiao
- lixsh6
- pinxuezhao
- QAQdevWestlake University
- SeTrionesBeijing
- ShenglongZ
- SimphoniTsinghua University
- Sun2018421China University of Petroleum,Beijing
- wang-zeruiShanghai AI Laboratory
- Yightwing
- youhebuke
- Youngluc
- yuantailingBeijing, China
- yuyq96HUAWEI
- zheng-kuaishou
- zzxzzx123UESTC