This is the code for "The Evolution of Gradient Descent" by Siraj Raval on Youtube
This weeks coding challenge is to write out the Adam optimization strategy from scratch. In the process you'll learn about all the other gradient descent variants and why Adam works so well. Bonus points if you add a visual element to it by plotting it in a Jupyter notebook. Good luck!
This is the code for this videon on Youtube by Siraj Raval. In the video, we go over the different optimizer options that Tensorflow gives us. Under the hood, they are all variants of gradient descent.
- matplotlib
- pyplot
- numpy
install missing dependencies with pip
Run jupyter notebook to see the code that compares gradient descent to stochastic gradient descent run in the browser. I've also got 2 seperate python files, one for adadelta and one for the nesterov method. Run those straight from terminal with the python command.
The credits for this code go to GRYE and dtnewman. I've merely created a wrapper to get people started.