/Adaptive-Nonconvex-Optimization

Implementation of YOGI optimizer

Primary LanguagePythonGNU General Public License v3.0GPL-3.0

Adaptive Methods for Nonconvex Optimization.

Implementation of YOGI optimizer as mentioned in the paper https://papers.nips.cc/paper/8186-adaptive-methods-for-nonconvex-optimization.

Getting Started

Yogi is an optimizer proposed to overcome the disadvantages of Adam Optimizer.

Implementation

This is the first implementaion of Yogi Optimizer in PyTorch.

How to Run

Refer to PyTorch folder README for running the code.

To-Do

  1. Implement Yogi in

    a) MxNet

    b) Tensorflow

  2. Add more experiments such as:

    a) Auto Encoders

    b) Machine Translation