/Gradient-descent-R-demo

Implementation of vanilla gradient descent, Newton’s method, damped Newton’s method, conjugate gradient descent method, momentum, Nesterov accelerated gradient, Adagrad, Adadelta/RMSprop, and Adam.

Primary LanguageR

  • vanilla gradient descent
  • Newton’s method
  • damped Newton’s method
  • conjugate gradient descent
  • momentum
  • Nesterov accelerated gradient
  • Adagrad
  • Adadelta/RMSprop
  • Adam

Functions (optimization and visualization) are all implemented in gradientDescent.R . To run the demo, use RStudio to see the visualized graphs and charts, and tweak the parameters.