Stack-augmented Parser-Interpreter Neural Network

This repository contains the source code based on the paper A Fast Unified Model for Sentence Parsing and Understanding and original codebase. For a more informal introduction to the ideas behind the model, see this Stanford NLP blog post.

The included implementations are:

A Python/Chainer implementation of SPINN using a naïve stack representation (named fat-stack)

Python code

The Python code lives, quite intuitively, in the python folder. We used this code to train and test the SPINN models before publication.

Installation

Requirements:

Python 2.7
Chainer 1.17
CUDA >= 7.0
CuDNN == v4 (v5 is not compatible with our Theano fork)

Install all required Python dependencies using the command below.

pip install -r python/requirements.txt

Running the code

The main executable for the SNLI experiments in the paper is fat_classifier.py, whose flags specify the hyperparameters of the model. You can specify the gpu id using the --gpu <gpu_id> flag. Uses the CPU by default.

Here's a sample command that runs a fast, low-dimensional CPU training run, training and testing only on the dev set. It assumes that you have a copy of SNLI available locally.

PYTHONPATH=spinn/python \
    python2.7 -m spinn.models.fat_classifier --data_type snli \
    --training_data_path snli_1.0/snli_1.0_dev.jsonl \
    --eval_data_path snli_1.0/snli_1.0_dev.jsonl \
    --embedding_data_path spinn/python/spinn/tests/test_embedding_matrix.5d.txt \
    --word_embedding_dim 5 --model_dim 10

For full runs, you'll also need a copy of the 840B word 300D GloVe word vectors.

Viewing Summaries in Tensorboard

To view some statistics in Tensorboard, make sure to turn the "write_summary" flag on. In other words, your run command should look something like this:

PYTHONPATH=spinn/python \
    python2.7 -m spinn.models.fat_classifier --data_type snli \
    --training_data_path snli_1.0/snli_1.0_dev.jsonl \
    --eval_data_path snli_1.0/snli_1.0_dev.jsonl \
    --embedding_data_path spinn/python/spinn/tests/test_embedding_matrix.5d.txt \
    --word_embedding_dim 5 --model_dim 10 \
    --write_summaries True

You'll also need to install Tensorflow.

Data

To download SST:

curl -o temp.zip http://nlp.stanford.edu/sentiment/trainDevTestTrees_PTB.zip
unzip temp.zip
rm temp.zip
mv trees sst

sed -i -e 's/)/\ )/g' ./sst/dev.txt
sed -i -e 's/)/\ )/g' ./sst/test.txt
sed -i -e 's/)/\ )/g' ./sst/train.txt

Creating Binary SST data:

python python/spinn/data/sst/load_sst_data.py binary

License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use these files except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.