Feature request: Please, add implementation for Novograd algorithm

[LSinev]

Great work!

Can you, please, add implementation for Novograd algorithm?
Support info:
paper: https://arxiv.org/abs/1905.11286

Novograd implementations:
https://github.com/NVIDIA/apex/blob/master/apex/optimizers/fused_novograd.py
https://github.com/jettify/pytorch-optimizer/blob/master/torch_optimizer/novograd.py
https://github.com/convergence-lab/novograd
https://github.com/lonePatient/NovoGrad-pytorch
https://github.com/titu1994/keras_novograd

[TimDettmers]

I had a look before at this. It is a bit more work and I will probably not focus on it for the next release, but it's good to know that people are interested in this (I had the request before). I will probably implement it sometime next month.