A sentiment analyzer using deep rnns, built with TensorFlow.
- Python 2 or 3
- Install [TensorFlow 1.2]https://www.tensorflow.org/install/)
$ pip install nltk
Next you will need to download the nltk tokenizer dataset:
- run a python session in terminal
$ python >>> import nltk>>> nltk.download()- type 'd', then hit enter
- type 'punkt' then hit enter
You can exit the python terminal once the download is done, your dependencies will be all setup.
To run with preset hyper-parameters just run:
$ python train.py
To try other hyper-parameters, you can change them in the config file. If you want to provide your own:
$ python train.py --config_file="path_to_config"
Descripton of hyper parameters:
| Name | Type | Description |
|---|---|---|
| hidden_size | int | number of hidden units in hidden layers |
| num_layers | int | number of hidden layers |
| batch_size | int | size of batchs in training |
| max_epoch | int | max number of epochs to train for |
| learning_rate | float | beggining learning rate |
| steps_per_checkpoint | int | number of steps before running test set |
| lr_decay_factor | float | factor by which to decay learning rate |
| batch_size | int | size of batchs in training |
| max_seq_length | int | maximum length of input token sequence |
| checkpoint_dir | string | directory to store/restore checkpoints |
| dropout | float | probability of hidden inputs being removed |
| grad_clip | int | max gradient norm |
When you first run train.py, it will download the data set, unzip, and process it. This will take several minutes. The training will begin automatically after this is done.
After your model is trained you can run:
$ python sample.py --text="Your text to sample here"
It will return 0 or 1. This number corresponds to a positive (1), or a negative (0) score.
The model is: embedding layer -> LSTMCells -> logistic regression output layer. I'll provide a picture in the future.
The hidden state averaged across all time steps is what is passed to the logistic regression layer.
I've begun to implement tensorboard variables and histograms.
You can access tensorboard with this application by using:
$ tensorboard --logdir=/tmp/tb_logs/
Then copy and paste the localhost url your terminal window gives you, into your browser of choice.
There isn't much to see right now except an overview of the graph. I will include screenshots of this down the road as I add more tensorboard functionality to this project.
Pre-trained model will be committed soon. Currently with defaults, it gets about 70%. I would like to get it to about 80-85% before committing the model.
Thanks to reddit user /u/LeavesBreathe for the help with getting the last hidden state
The dataset used was the Large Movie Review Dataset.
I also referred to this tutorial as a starting point