LiyuanLucasLiu/RAdam

Issues

• simplify add_

  #60 opened 4 years ago by LucasMourot
  1

• NaNs

  #61 opened a year ago by thegodone
  4

• Does RAdam usually need an annealing and warm up scheduler?

  #68 opened 2 years ago by brando90
  2

• Question regarding 2nd Moment Update

  #69 opened 3 years ago by thisisbowen
  1

• Is RAdam needed when fitting perfectly a small batch e.g. 500 examples?

  #63 opened 3 years ago by brando90
  3

• Are the plots you have wrt epochs or iterations?

  #67 opened 3 years ago by brando90
  1

• How to choose decay rate? (No success with RAdam - does one need a decay scheduler or gradient clipping)

  #66 opened 3 years ago by brando90
  5

• Overload of addcmul_ is deprecated:

  #49 opened 4 years ago by sooheon
  2

• RAdam for pytorch official

  #62 opened 3 years ago by brando90
  6

• Should one be using RAdam or PlainRadam?

  #65 opened 3 years ago by brando90
  1

• Question of RAdam's dependence on the number of examples

  #64 opened 3 years ago by brando90
  1

• Will radam be affacted by weight decay?

  #59 opened 4 years ago by CoinCheung
  0

• Deprecated Warning in `RAdam` with torch==1.7.1

  #58 opened 4 years ago by wenmin-wu
  2

• Any concern for using `math.sqrt` instead of `torch.sqrt`

  #57 opened 4 years ago by wenmin-wu
  2

• Why there are 10 slots in the buffer?

  #56 opened 4 years ago by nihil-admirari
  1

• RAdam Instability vs AdamW / Adam

  #54 opened 4 years ago by danielhanchen
  8

• Algorithm 2 Arxiv paper 1/beta2 typo?

  #55 opened 4 years ago by danielhanchen
  2

• Hi

  #53 opened 4 years ago by ARWEJS
  1

• Cannot reproduce the PPL on One Billion Words

  #50 opened 4 years ago by XuezheMax
  1

• distributed training generating "exp_avg error"

  #27 opened 4 years ago by h-jia
  2

• ResNet56

  #10 opened 5 years ago by Slawlight
  4

• What's the difference between RAdam and PlainRAdam?

  #48 opened 4 years ago by seraphzl
  1

• KeyError: 'buffer'

  #47 opened 5 years ago by MaNatsu8023
  1

• how can i use this in tf1.4

  #46 opened 5 years ago by HouGall
  0

• About the estimation of DoF

  #45 opened 5 years ago by omiita
  1

• Typo in paper

  #32 opened 5 years ago by zzaebok
  1

• erro: NameError: name 'iter_idx' is not defined,when i use AdamW

  #40 opened 5 years ago by daixiangzi
  2

• Can you make radam alone installable via pip?

  #42 opened 5 years ago by bwang-delft
  1

• TypeError: must be real number, not NoneType

  #38 opened 5 years ago by Fangyh09
  1

• Sensitivity wrt LR restarts

  #8 opened 5 years ago by depthwise
  12

• Why it have convertion to fp32?

  #35 opened 5 years ago by hadaev8
  1

• Having issues importing to a Kaggle notebook

  #31 opened 5 years ago by spencerkraisler
  1

• math.sqrt gets a negative argument

  #30 opened 5 years ago by akhileshgotmare
  5

• Theory question on warmup

  #22 opened 5 years ago by OverLordGoldDragon
  4

• adam-2k

  #25 opened 5 years ago by huoxuelu
  1

• Does RAdam break training with different learning rates for different param_groups?

  #24 opened 5 years ago by sholderbach
  3

• Speed performance

  #21 opened 5 years ago by ivanvovk
  5

• Could you share the tensorflow implementations?

  #18 opened 5 years ago by Pro-flynn
  2

• Notebook tutorial

  #16 opened 5 years ago by alexandreCameron
  7

• Become very unstable in BERT+MultiTask mode

  #20 opened 5 years ago by tanaka-jp
  1

• Different implementation of radam.py

  #19 opened 5 years ago by tanaka-jp
  1

• keras如何使用此优化器，能否给一个例子

  #12 opened 5 years ago by zxzxzxygithub
  1

• Could you pls give me some clues on the hyperparameters for ImageNet training?

  #11 opened 5 years ago by iamweiweishi
  1

• "Please see the Training recipes for how to train the models."

  #5 opened 5 years ago by bluesky314
  0

• Does RAdam have a Keras version？

  #3 opened 5 years ago by xingyi-li
  8

• [AdamW] amsgrad issue

  #9 opened 5 years ago by frgfm
  1

• Worse performance

  #7 opened 5 years ago by Slawlight
  4

• amsgrad is not defined in AdamW class

  #6 opened 5 years ago by piresramon
  2

• Please add the license

  #1 opened 5 years ago by depthwise
  1