The implementation of AMSGrad which is proposed by On the Convergence of Adam and Beyond(ICLR2018) in Theano.
You can carry out the comparison experiment between Adam and AMSGrad as following command:
THEANO_FLAGS=mode=FAST_RUN,device=cpu,floatX=float32 python3 adam_vs_amsgrad.py