MrTF/visualize_optimizers

Visualization of DL optimization algorithms

Visualization of various Deep Learning optimization algorithms on some set of optimization test functions. Algorithms are implemented in NumPy from scratch.

Algorithms

1. SGD
2. Adagrad
3. Adadelta
4. RMSProp
5. Adam
6. AdaMax
7. AdamW (WIP)

Examples

REFERENCES

• Animation code from: Louis Tiao
• An overview of gradient descent optimization algorithms by Sebastian Ruder