xrsrke/pipegoose

Reproducible in 3D Parallelism

xrsrke opened this issue a year ago · 0 comments

xrsrke commented a year ago

Reading

https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/tensor_parallel/random.py