Language modeling stuff has been moved to a new repo (it didn't really match with this one anymore, and I wanted to get my commit rate up).
Some RNN implementations in Theano, along with various optimizers. In particular:
vanilla.py: Basic RNN implementationgru.py: Gated recurrent unitlstm.py: Long short term memoryoptimizers.py:rmsprop,sgd, andmomentum(including Nesterov momentum)
Each file has a framework for generating random data and fitting the RNN model to that data, so you can play around with it.