Azure/MS-AMP

Issues

• MNIST single GPU example: GradScaler AssertionError

  #168 opened 7 months ago by 152334H
  1

• Optimizer compilation fails with PyTorch 2.2

  #158 opened 8 months ago by rosario-purple
  2

• MS-AMP crashes with DeepSpeed ZeRO 3

  #130 opened 2 months ago by rationalism
  4

• V0.4 Release Plan

  #123 opened a year ago by cp5555
  1

• Does this actually work?

  #178 opened 2 months ago by tsengalb99
  10

• Optimizer datatype

  #170 opened 2 months ago by brianchmiel
  4

• Questions about error reporting

  #127 opened 2 months ago by Mrzhang-dada
  2

• Is activation checkpointing used for Table 5 from the FP8-LM paper?

  #167 opened 2 months ago by SolitaryThinker
  2

• [Question] How to apply MS-AMP to only part of the model?

  #163 opened 2 months ago by veritas9872
  3

• Support for latest Megatron-LM and transformer-engine 1.0 +

  #139 opened 2 months ago by sosofun
  2

• question about the paper

  #125 opened 2 months ago by WeiSQ-zju
  4

• Support for MS-AMP in FSDP

  #122 opened 2 months ago by naveenkumarmarri
  3

• Questions: Clarifying the use of FP8 for Training

  #99 opened 2 months ago by jon-chuang
  2

• AttributeError: 'ScalingTensor' object has no attribute 'view'

  #180 opened 2 months ago by LSC527
  3

• Integration with PyTorch Lightning

  #179 opened 2 months ago by schopra8
  0

• Request for Update to Support Latest Megatron-LM Version

  #177 opened 3 months ago by nogizakar
  0

• Can I use fp8 only when the code runs to the fp8 branch?

  #176 opened 5 months ago by forevergj
  8

• Why does using msamp decrease throughput

  #175 opened 5 months ago by forevergj
  4

• [compilation error] nvcc fatal : Unsupported gpu architecture 'compute_89'

  #174 opened 5 months ago by fmo-mt
  0

• Clarification: do we need 20 or 16 bytes per parameter when training with Adam + Mixed precision

  #173 opened 5 months ago by rodrigo-f-nogueira
  2

• Please update obsolete dependencies

  #129 opened 7 months ago by rosario-purple
  7

• Optimized model seems slower than original

  #172 opened 6 months ago by BitCircuit
  3

• how can i export the model from pytorch to onnx?

  #171 opened 6 months ago by 221588
  1

• Qusetion: FP8 Allreduce

  #111 opened a year ago by MARD1NO
  7

• add topic tag mixed-precision

  #164 opened 7 months ago by Beliavsky
  1

• [Question]Is MS-AMP going to support ZeRO-2 + PP ?

  #154 opened 8 months ago by ohwi
  1

• MS-AMP install from source

  #133 opened 9 months ago by wpf19911118
  2

• Huggingface Accelerate Support

  #128 opened 9 months ago by muellerzr
  2

• Is MS-AMP reproducing the FP8-LM paper's results?

  #147 opened 9 months ago by xrsrke
  2

• Question about FP8 matmul coverage in FP8-LM

  #146 opened 9 months ago by stakahashy
  2

• Question: Is FP8-LM only supported on H100?

  #116 opened a year ago by LSC527
  7

• nccl buildig failed without specifying NVCC_GENCODE

  #44 opened a year ago by tocean
  3

• V0.3.0 Test Plan

  #107 opened a year ago by tocean
  0

• V0.3 Release Plan

  #92 opened a year ago by cp5555
  0

• FP8 in tensor parallel region question

  #119 opened a year ago by afcruzs
  4

• FP8 in linear layer question

  #118 opened a year ago by afcruzs
  2

• Automatic Scaling in the code

  #117 opened a year ago by afcruzs
  2

• Question: Difficulty of FP8 + ZeRO

  #108 opened a year ago by awgu
  1

• Training curve datapoints or smoothing

  #115 opened a year ago by afcruzs
  1

• Question : does it work with Apple mps ?

  #110 opened a year ago by edmondja
  2

• Replace dist_op with fp8_op

  #86 opened a year ago by tocean
  0

• V0.2 Release Plan

  #67 opened a year ago by cp5555
  0

• V0.2.0 Test Plan

  #87 opened a year ago by tocean
  0

• `make postinstall` failed due to undeclared symbols.

  #88 opened a year ago by guoshzhao
  4

• unit-test for multi-process training

  #61 opened a year ago by wkcn
  0

• Support FP8 ProcessGroup in pytorch

  #50 opened a year ago by tocean
  0

• [Bug] `LBOptimizer.all_reduce_grads` reduces gradients of only a model, even if training several models

  #62 opened a year ago by wkcn
  0

• MNIST example failed in docker nvcr.io/nvidia/pytorch:22.09-py3

  #53 opened a year ago by tocean
  0

• V0.1.0 Test Plan

  #51 opened a year ago by tocean
  1

• Can not run mnist_ddp.py when using pytorch 1.14

  #49 opened a year ago by tocean
  2