gpauloski/kfac-pytorch

Issues

• Is this implementation of the kfac algorithm compatible with the nn module which is not a distributed module?

  #150 opened 9 months ago by zhangylch
  4

• Adaptive damping

  #113 opened a year ago by phuvm72
  1

• Using K-FAC training BERT-Large model

  #117 opened a year ago by RuidongYan666
  3

• Introduction of RoBERTa Pretraining (in BERT-PyTorch repo.)

  #124 opened a year ago by RuidongYan666
  1

• This code uses the empirical fisher matrix instead of actual fisher matrix.

  #109 opened a year ago by siddharth9820
  2

• Create wiki

  #39 opened 2 years ago by gpauloski
  1

• Use ncclAvg reduce op instead of future hook that divide tensor by world size

  #34 opened 3 years ago by gpauloski
  0

• Add Apex as optional requirement

  #51 opened 2 years ago by gpauloski
  0

• Use allreduce_coalesced for factor allreduce

  #35 opened 3 years ago by gpauloski
  1

• Weird Behavior With 2 GPUs and mpiexec // horovorun

  #92 opened 2 years ago by ConstantinPuiu
  2

• The problem of this new edition project

  #86 opened 2 years ago by Elec-coder
  2

• Add `torch` as mypy additional dependency

  #59 opened 2 years ago by gpauloski
  0

• Regression in Layer Registration Recursion

  #53 opened 2 years ago by gpauloski
  0

• Running problems

  #57 opened 2 years ago by Elec-coder
  2

• Running Problems

  #54 opened 2 years ago by Elec-coder
  3

• Print env info in example scripts

  #44 opened 3 years ago by gpauloski
  0

• Remove tags in titles in issue templates

  #45 opened 3 years ago by gpauloski
  0

• Add BatchNorm support

  #23 opened 4 years ago by gpauloski
  0

• Add diagonal approximation for linear layers

  #26 opened 4 years ago by gpauloski
  0

• Add Tensor Communication Bucketing

  #32 opened 3 years ago by gpauloski
  0

• KFAC verbose should use logging instead of print

  #33 opened 3 years ago by gpauloski
  0

• Add gate support to linear multi layers

  #24 opened 3 years ago by gpauloski
  1

• Update backward hooks for PyTorch 1.8 compatibility

  #29 opened 3 years ago by gpauloski
  1

• When i run the example tran_imagenet_resnet.py by pytorch, dist.broadcast: RuntimeError: Tensors must be contiguous

  #31 opened 4 years ago by JLtwoP
  3

• Test KFAC load/save state dict

  #27 opened 4 years ago by gpauloski
  0

• Investigate embedding layer performance.

  #25 opened 4 years ago by gpauloski
  1

• Support factor_decay=1

  #20 opened 4 years ago by gpauloski
  1

• Concat Eigenvalues to Eigenvectors when communicating

  #18 opened 4 years ago by gpauloski
  1

• Investigate time-per-epoch slow down over course of training

  #17 opened 4 years ago by gpauloski
  1

• Test broadcast eigens instead of allreduce(sum)

  #14 opened 4 years ago by gpauloski
  0

• Change round-robin layer placement to optimize by layer size

  #13 opened 4 years ago by gpauloski
  0

• Add gradient accumulation to imagenet example

  #22 opened 4 years ago by gpauloski
  1

• add lr tensorboard logging to examples

  #8 opened 5 years ago by gpauloski
  0

• Add finer block diagonal approx to `optimized` branch

  #12 opened 5 years ago by gpauloski
  0

• Add damping decay

  #15 opened 5 years ago by gpauloski
  0

• Add kfac update freq decay

  #16 opened 5 years ago by gpauloski
  0

• Remove redundant communication for kFAC supported layers when reducing non-KFAC supported layers

  #9 opened 5 years ago by gpauloski
  1

• Change cp imagenet script to just copy data not tar then extract

  #6 opened 5 years ago by gpauloski
  1

• Check if all KFAC tensors in cuda mem

  #4 opened 5 years ago by gpauloski
  1

• Make current KFAC impl into a gradient preconditioner

  #2 opened 5 years ago by gpauloski
  0

• Fix training scripts

  #3 opened 5 years ago by gpauloski
  0

• Cite repos I found resnet models from

  #1 opened 5 years ago by gpauloski
  0