zane-star-bot/Optimizer-Visualization

2019 Fall semester: Dynamic Simulations class. Visualize optimizers using MATLAB - GD, SGD, Momentum, Adagrad, Adadelta, RMSProp, Adam, NAdam, RAdam

Optimizer Visualization

Visualization of Optimizers.

Optimizers

Our supporting optimizers.

From GD to NAdam are referenced (arxiv 1609.04747 - Understanding Gradient Descent Algorithm)

• GD (Gradient Descent)
• SGD (Stochastic Gradient Descent)
• Momentum
• NAG (Nesterov accelerated gradient)
• Adagrad
• AdaDelta
• RMSProp
• Adam
• AdaMax
• NAdam
• RAdam (arxiv 1908.03265 - On the Variance of the Adaptive Learning Rate and Beyond)

If you want any more optimizers to add here, please write on issues tab.

GUI

Test on GUI is available.

How to run

Open newoptimizer.fig with MATLAB GUIDE and run.