xrsrke/pipegoose

Issues

• Automatic module mapping using torch.fx

  #40 opened a year ago by xrsrke
  3

• Multimodal MoE

  #61 opened a year ago by xrsrke
  0

• DiLoCo replication (DiLoCo: Distributed Low-Communication Training of Language Models)

  #59 opened a year ago by xrsrke
  0

• Implement new pipeline parallelism technique

  #7 opened a year ago by xrsrke
  0

• End-to-end FP8 training

  #45 opened a year ago by xrsrke
  1

• Port CUDA Kernels

  #8 opened a year ago by xrsrke
  6

• Save and load checkpoints

  #29 opened a year ago by xrsrke
  0

• Distributed CLIP

  #60 opened a year ago by xrsrke
  0

• Making pipeline parallelism compatible with `transformers`

  #52 opened a year ago by xrsrke
  0

• Callbacks for Distributed Optimizer

  #21 opened a year ago by xrsrke
  0

• Gradient Checkpointing

  #4 opened a year ago by xrsrke
  1

• Mixture of Experts

  #19 opened a year ago by xrsrke
  0

• Kernel Fusion using torch.jit

  #10 opened a year ago by xrsrke
  3

• Deparallelize tensor parallelism

  #11 opened a year ago by xrsrke
  4

• Deparallelize pipeline parallelism

  #34 opened a year ago by xrsrke
  0

• Distributed Logger

  #33 opened a year ago by xrsrke
  1

• Tensor Parallelism

  #37 opened a year ago by 3outeille
  0

• Lazy initialization of massive models

  #25 opened a year ago by xrsrke
  1

• Reproducible in 3D Parallelism

  #15 opened a year ago by xrsrke
  0

• Mixed precision training in FP16

  #14 opened a year ago by xrsrke
  0

• Trainer

  #18 opened a year ago by xrsrke
  4

• Fused Optimizer

  #13 opened a year ago by xrsrke
  8

• Model partitioning for pipeline parallelism

  #6 opened a year ago by xrsrke
  1

• Dataloader and Sampler for 3D Parallelism

  #9 opened a year ago by xrsrke
  0

• Bucket small tensors and collective operations into larger ones

  #5 opened a year ago by xrsrke
  0

• ZeRO-1

  #20 opened a year ago by xrsrke
  0

• Setup documentation

  #16 opened a year ago by xrsrke
  0

• Sequence Parallelism

  #22 opened a year ago by xrsrke
  1

• Checkpointing

  #24 opened a year ago by xrsrke
  4

• Support TPU

  #26 opened a year ago by xrsrke
  0

• Support parallelizing arbitrary transformer torch modules

  #27 opened a year ago by xrsrke
  0

• Implement new tensor parallelism technique

  #17 opened a year ago by xrsrke
  0