konstmish/prodigy

Issues

• [Question] Limit learning rate

  #37 opened 21 days ago by Specthram
  10

• lora training with fsdp will hang up

  #35 opened 3 months ago by gameofdimension
  0

• Can we combine this profitably with Schedule-Free?

  #17 opened a year ago by JDvorak
  5

• Prodigy 8-bit version

  #21 opened 6 months ago by KonokoAz
  2

• Question on convergence

  #18 opened a year ago by ppbrown
  5

• Lowering TE or Unet average only

  #16 opened 5 months ago by trihardseven
  2

• Prodigy is not working well with Stable Diffusion 3.5 Medium training

  #27 opened 5 months ago by Bocchi-Chan2023
  4

• Recommended Prodigy Settings (low steps per epoch)

  #8 opened a year ago by brandostrong
  9

• beta3: difference between paper and code

  #24 opened 5 months ago by dxqbYD
  1

• Growth_rate

  #15 opened 6 months ago by DarkAlchy
  2

• Possible to marry Prodigy and AdamW?

  #11 opened a year ago by askerlee
  8

• Document incompatibility with gradient clipping

  #13 opened a year ago by crypdick
  2

• Is there a way to monitor the estimated LR over time, if it has any meaning?

  #12 opened a year ago by ethansmith2000
  1

• Is Prodigy compatible with Pytorch's automatic mixed precision?

  #14 opened a year ago by crypdick
  0

• T_MAX value (CosineAnnealingLR)

  #10 opened a year ago by josemerinom
  3

• Question

  #3 opened a year ago by DarkAlchy
  38

• Question regarding t_max and d estimation

  #6 opened a year ago by DanPli
  1

• Is the any rule of the thumb for tuning weight_decay of Prodigy when training transformers-based LLMs?

  #5 opened 2 years ago by DesperateExplorer
  3

• Question

  #4 opened 2 years ago by Dentoty
  3

• module 'torch.optim' has no attribute 'Prodigy'

  #2 opened 2 years ago by DarkAlchy
  4

• RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

  #1 opened 2 years ago by manyotherfunctions
  1