A simple vanilla implementation of a recurrent LSTM neural network implemented in python
A straightforward implementation of an LSTM neural network for comments for clarity. It includes a manual implementation of RMSprop.
This was part of a small project of mine to try to better understand how neural networks work. The code is functional, however it is slow and difficult to train, often failing to converge or overshooting the minimum. I believe that learning rate decay may improve it's performance but I have been unable to implement it successfully. I may return to try again when I know more.
In the meantime if anybody improves on this model or works out effective hyperparameters I would be very grateful to hear about it.