lessw2020/Ranger21

Issues

• lr below min_lr check too aggressive

  #16 opened 4 years ago by kai-tub
  2

• Error when using DDP

  #49 opened 2 years ago by smartbarbarian
  1

• error in warmdown - lr below min lr. current lr = 2.999999999999997e-0518 [07:50<00:04, 4.66s/it] auto handling but please report issue!

  #28 opened 3 years ago by neuronflow
  2

• learning rate scheduler

  #48 opened 2 years ago by shyhyawJou
  0

• Recommended settings for transformers?

  #47 opened 2 years ago by OhadRubin
  1

• Require an documentation

  #38 opened 3 years ago by huangnengCSU
  2

• What it the best hyper-parameter setting?

  #42 opened 3 years ago by NoOneUST
  2

• sample usage in fastai

  #35 opened 3 years ago by nikky4D
  1

• hi，please help me

  #46 opened 2 years ago by wudizuixiaosa
  0

• resuming training with ranger21?

  #18 opened 4 years ago by neuronflow
  3

• Nice name of your project)

  #45 opened 3 years ago by Ranger21
  0

• AttributeError: module 'collections' has no attribute 'Callable'

  #43 opened 3 years ago by JackKelly
  0

• Gradient normalization lowers the maximum learning rate that can converge.

  #41 opened 3 years ago by Handagot
  0

• Can ranger be used for NLP transformers?

  #40 opened 3 years ago by LifeIsStrange
  2

• Not support pytorch _1.3.1

  #39 opened 3 years ago by huangnengCSU
  0

• RuntimeError: hit nan for variance_normalized

  #30 opened 3 years ago by gcp
  7

• decouple the lr scheduler and optimizer?

  #36 opened 3 years ago by hiyyg
  5

• hit nan for variance_normalized

  #19 opened 4 years ago by jimmiebtlr
  7

• local variable 'neg_grad_ma' referenced before assignment when momentum_type is not "pnm"

  #34 opened 3 years ago by lechmazur
  0

• About gradient normalization

  #29 opened 3 years ago by julightzhong10
  0

• Multi GPU problem

  #17 opened 4 years ago by zsgj-Xxx
  4

• Performance of ResNet50 on ImageNet

  #27 opened 3 years ago by juntang-zhuang
  1

• SAM paper

  #25 opened 3 years ago by ryanstout
  0

• File "/home/.../site-packages/ranger21/ranger21.py", line 680, in step raise RuntimeError("hit nan for variance_normalized")

  #24 opened 3 years ago by neuronflow
  1

• comparing ranger21 to SAM optimizer

  #21 opened 3 years ago by nikky4D
  2

• error when training with batch_size = 1

  #20 opened 3 years ago by neuronflow
  0

• optimizer = Ranger21(params=model.parameters(), lr=learning_rate) File "/mnt/Drive1/florian/msblob/Ranger21/ranger21/ranger21.py", line 179, in __init__ self.total_iterations = num_epochs * num_batches_per_epoch TypeError: unsupported operand type(s) for *: 'NoneType' and 'NoneType'

  #12 opened 4 years ago by neuronflow
  6

• adaptive gclipping: unable to process len of 5 - currently must be <= 4

  #11 opened 4 years ago by HungYu-Wu
  6

• Adaptive Gradient Clipping

  #10 opened 4 years ago by benihime91
  3

• Changes in lr

  #9 opened 4 years ago by zsgj-Xxx
  7

• torch.grad removed in PyTorch 1.8.1?

  #8 opened 4 years ago by jszym
  2

• Augmentation requests

  #5 opened 4 years ago by LifeIsStrange
  2

• Example

  #3 opened 4 years ago by johnyquest7
  1

• Adaptive Gradient Clipping

  #1 opened 4 years ago by kayuksel
  2