kozistr/pytorch_optimizer

Issues

• Support Adopt

  #289 opened 2 months ago by sdbds
  0

• schedulefree with palm schedule

  #286 opened 2 months ago by sdbds
  0

• Add torch._foreach_ for speed up

  #287 opened 2 months ago by sdbds
  0

• Request to add 4-bit AdamW

  #208 opened 2 months ago by LiutongZhou
  4

• SophiaH implementation is not correct

  #278 opened 2 months ago by Vectorrent
  4

• sophiah in https://github.com/booydar/LM-RMT

  #194 opened 2 months ago by robotzheng
  8

• 'grokfast_after_step' argument causes the optimizer to crash

  #279 opened 2 months ago by Vectorrent
  1

• Support SOAP

  #274 opened 3 months ago by sdbds
  0

• Support SaRA

  #273 opened 3 months ago by sdbds
  0

• Add WeLore

  #262 opened 4 months ago by rotem154154
  2

• The `create_optimizer()` function fails when Lookahead arguments are provided

  #270 opened 4 months ago by Vectorrent
  1

• Prodigy optimizer cannot be resumed from checkpoint

  #266 opened 4 months ago by Vectorrent
  4

• AdamG: Towards Stability of Parameter-free Optimization

  #264 opened 4 months ago by Vectorrent
  6

• Add TRAC

  #256 opened 5 months ago by sdbds
  0

• Add Kate

  #249 opened 6 months ago by tfriedel
  6

• Add Adam-Mini

  #246 opened 6 months ago by sdbds
  1

• Add StableAdamW

  #250 opened 6 months ago by tfriedel
  0

• Add WSD LR scheduler

  #247 opened 6 months ago by sdbds
  0

• GrokFast: Accelerated Grokking by Amplifying Slow Gradients

  #244 opened 6 months ago by redknightlois
  0

• Modified AdaFactor by ViT paper

  #236 opened 7 months ago by aliencaocao
  1

• FAdam

  #241 opened 7 months ago by tranquan687
  1

• Lookahead`s param_groups attribute is not loaded from checkpoint.

  #237 opened 7 months ago by michaldyczko
  0

• Wrong typing of `reg_noise`

  #239 opened 7 months ago by michaldyczko
  0

• Plans for pytorch_optimizer v3

  #164 opened 7 months ago by kozistr
  1

• Ranger sign inversion

  #232 opened 8 months ago by i404788
  1

• ScheduleFree

  #230 opened 8 months ago by sdbds
  0

• Entropy-MCMC: Sampling from flat basins with ease

  #231 opened 8 months ago by redknightlois
  0

• Improvement to SAM: SAM as an Optimal Relaxation of Bayes

  #212 opened 8 months ago by redknightlois
  1

• Adalite

  #225 opened 9 months ago by sdbds
  1

• GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

  #224 opened 9 months ago by redknightlois
  1

• Aida optimizer

  #220 opened 10 months ago by okbalefthanded
  3

• [Feature request]REX LR scheduler

  #217 opened 10 months ago by sdbds
  2

• FR: Sharpness-Aware Minimization Revisited: Weighted Sharpness as a Regularization Term (WSAM)

  #213 opened a year ago by LiutongZhou
  2

• Ranger21 has undocumented required arguments

  #214 opened a year ago by Vectorrent
  3

• ipex failed for Adan from pytorch_optimizer

  #210 opened a year ago by ldv1
  1

• Empty Docs Sections

  #204 opened a year ago by InfluxOW
  6

• Lookahead is not a subclass of torch.optim.Optimizer

  #200 opened a year ago by georg-wolflein
  4

• Adding the CAME optimizer

  #195 opened a year ago by LiutongZhou
  2

• sophiah bug

  #193 opened a year ago by robotzheng
  5

• Can a variant of Lion named Tiger be added to your package?

  #191 opened a year ago by liujuncn
  2

• LOMO: LOw-Memory Optimization

  #187 opened 2 years ago by sdbds
  1

• Prodigy: An Expeditiously Adaptive Parameter-Free Learner

  #182 opened 2 years ago by sdbds
  2

• get_chebyshev_schedule not working

  #168 opened 2 years ago by aliencaocao
  2

• Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training

  #173 opened 2 years ago by redknightlois
  2

• Ranger21 causing loss to spike and model never converges

  #169 opened 2 years ago by aliencaocao
  8

• Question about using Ranger21 with Hugging Face Trainer

  #167 opened 2 years ago by aliencaocao
  4

• Implement optimizers

  #152 opened 2 years ago by kozistr
  0

• Implement optimizers

  #138 opened 2 years ago by kozistr
  0

• RotoGrad: Gradient Homogenization in Multitask Learning

  #124 opened 2 years ago by redknightlois
  1

• Adafactor: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

  #131 opened 2 years ago by Bing-su
  3