/stacked-RNN-Numpy

Character-level Language Model with stacked RNN (ONLY Numpy)

Primary LanguagePythonMIT LicenseMIT

Character-level Language Model with stacked RNN (ONLY Numpy)

I implemented class module for stacked Recurrent Neural Networks with ONLY Numpy package.

The following parameters can be selected in the RNN class.

  • input size
  • output size
  • hidden unit size
  • time length
  • depth size
  • batch size
  • dropout rate
  • learning rate

Additional information for RNN class,

  • Weight initialization : Xavier initialization
  • Weight update optimizer : Adagrad, RMSProp
  • Dropout

To test the stacked RNN model, I used text data in online.
It is for sequence generation similar with character-level language model.
You can find out the text data used for training in data directory.


Install environments

See requirements.txt

Select one of install methods below

  • Install all required packages with only one command line
    $ pip install --upgrade -r requirements.txt

  • Install required packages individually
    numpy == 1.17.4
    matplotlib == 3.1.1

If *.py file doesn't run after installing required packages, check 'My working environment' in requirements.txt


Source code

  • utils.py : Includes several necessary function for running the other source code
  • model.py : class RNN (stacked Recurrent Neural Networks) with ONLY Numpy
  • train.py : Train data with RNN class
  • test.py : Test file for live demo

If you want to check the training process, run train.py
If you want to check the final result, run test.py.

  • data : Text data that I used for training the RNN model
  • figure : Loss, Iteration graph
  • models : Pre-implemented short codes for RNN, LSTM
  • ppt : Presentation file with details for this project
  • result : Results file stored by RNN object format(.pickle)

Reference

https://github.com/janivanecky/Numpy-RNNs
https://gist.github.com/karpathy/d4dee566867f8291f086


License

MIT License