/keras-opt

Implementation of some new techniques from fastai and other papers which works with keras models

Primary LanguagePython

Keras OPT Helpers

Contains keras callbacks and optimizers for training keras models

LR Finder

Finds optimal learning rate for model - paper (section 3.3)

Schedulers

Contain following schedulers:

  • SGD with warm restarts - paper
  • Cyclic learning rates - paper

Optimizers

Contains optimizers from official keras repo added with some optimization techniques.

  • Weight decay
  • Discriminative learning rates

TODO

  • Weight decay normalization with wd_multi(below algo 2) and adam with restarts - paper