mikkelbrusen/awd-inspired-lstm

Notebook containing an implementation of a Language Model using state-of-the-art techniques also seen in the current state-of-the-art AWD-LSTM

awd-inspired-lstm

This repository contains an implementation of a Language Model using state-of-the-art techniques also seen in the current state-of-the-art AWD-LSTM

This model was built as part of a project in the course 02456 Deep Learning @ DTU - Technical University of Denmark

• This code was originally forked from the PyTorch word level language modeling example and is heavily inspired by the original AWD-LSTM implementation LSTM and QRNN Language Model Toolkit

• The notebook is available on google colab and in this github repo, but instructions to run will only be given for the google colab version, for which we give instructions to train a word level language models over the Penn Treebank (PTB)

Software Requirements

Python 3 and PyTorch 0.4 are required for the current codebase.

License

This project is licensed under the MIT License - see the LICENSE.md file for details