This repo provides the best MT-LSTM from the paper Learned in Translation: Contextualized Word Vectors (McCann et. al. 2017). For a high-level overview of why CoVe are great, check out the post.
example.py uses torchtext to load the Stanford Natural Language Inference Corpus and GloVe.
It uses a PyTorch implementation of the MTLSTM class in mtlstm.py to load a pretrained encoder, which takes in sequences of vectors pretrained with GloVe and outputs CoVe.
We have included a Dockerfile that covers all dependencies.
We typically use this code on a machine with a GPU,
so we use nvidia-docker.
Once you have installed Docker,
pull the docker image with docker pull bmccann/cove.
Then you can use nvidia-docker run -it -v /path/to/cove/:/cove cove to start a docker container using that image.
Once the container is running,
you can use nvidia-docker ps to find the container_name and
nvidia-docker exec -it container_name bash -c "cd cove && python example.py" to run example.py.
You will need to install PyTorch and then run pip install -r requirements.txt
Run the example with python example.py.
If using this code, please cite:
B. McCann, J. Bradbury, C. Xiong, R. Socher, Learned in Translation: Contextualized Word Vectors
@article{McCann2017LearnedIT,
title={Learned in Translation: Contextualized Word Vectors},
author={Bryan McCann and James Bradbury and Caiming Xiong and Richard Socher},
journal={arXiv preprint arXiv:1708.00107},
year={2017}
}
Contact: bmccann@salesforce.com