Natural Language Inference is fundamental to many Natural Language Processing applications such as semantic search and question answering. The task of NLI has gained significant attention in the recent times due to the release of fairly large scale, challenging datasets. Present approaches that address NLI are largely focused on learning based on the given text in order to classify whether the given premise entails, contradicts, or is neutral to the given hypothesis. On the other hand, techniques for Inference, as a central topic in artificial intelligence, has had knowledge bases playing an important role, in particular for formal reasoning tasks. While, there are many open knowledge bases that comprise of various types of information, their use for natural language inference has not been well explored. In this work, we present a simple technique that can harnesses knowledge bases, provided in the form of a graph, for natural language inference.
Before running the experiment for the first time, you will need to download and preprocess the datasets and the word embedding files.
To run the setup scripts, clone the repository and enter the ./scripts
git clone <to be replace by actual git url>
cd repo_root/entailment/scriptsThe following script will download and unzip the NLI datasets (SciTail, SNLI, MultiNLI) into ./data/nli_datasets
bash ./download_datasets.shAnd the following script will download and unzip the word embeddings (GloVe, ConceptNet-PPMI) into ./data/glove and ./data/conceptnet, respectively.
bash ./download_embeddings.shBefore installing the required packages, it is recommended that you
create a virtual environment dedicated for each of the project,
especially if you are using different versions of the packages. Some of
the popular options include conda,
pipenv, or the naive
virtualenv.
Most of the code are written in Python 3.6, but should be compatible
with other versions of Python.
Going back to the root directory of the project if you are still in the script directory.
cd ..Once you create and activate the environment, go ahead and install the packages
pip install -r requirements.txtDepending on the CUDA version (if any) and the operating system you are running, you might need to manually install the suitable PyTorch version from here.
TBD
We are using OpenKE to generate our knowledge graph embedding from ConceptNet. For details on how to setup the environment or modify configurations, please visit OpenKE's home page, or see the same README file imported in this repository.
In short, before training the embeddings for the first time, you will need to compile the model
cd kg-embeddings/OpenKE
bash make.shThen, you can move to the parent directory and begin training the embeddings
cd ..
python run.pyThe output of this script should be a 300d TransH embedding of
ConceptNet, which will be stored at ./data/embeddings/conceptnet/
as a binary file.
To convert that binary file into the format that can be read by AllenNLP later on, run the following script
python ./data_preprocessing/process-transh.pyAlso, the previously downloaded ConceptNet-PPMI embedding also needs to be process before it can be used by AllenNLP. To process ConceptNet-PPMI, run the following script:
python ./data_preprocessing/process-conceptnet-ppmi.pyOnce everything are set up, you can start training the models by running the following scripts
bash train.shThis will train all the models, log the relevant information to stdout,
and store the serialized models to ./res
To run each of the model individually, execute the following command:
allennlp train <model_config> --include-package code -s <stored_directory>e.g.:
allennlp train config/graph.json --include-package code -s res/graphYou can view the configuration files in the ./config/
directory. Here are the list of available models
| Model | Paper | Config |
|---|---|---|
| DecompAttn | A Decomposable Attention Model for Natural Language Inference (Parikh et al, 2016) | decompattn.json |
| MatchLSTM | Learning Natural Language Inference with LSTM (Wang and Jiang, 2016) | matchlstm.json |
| DeIsTe | End-Task Oriented Textual Entailment via Deep Explorations of Inter-Sentence Interactions (Yin et al, 2018) | deiste.json |
| SimpleGraph | N/A | graph.json |
| DecompAttn + Graph | N/A | decompattn_graph.json |
| MatchLSTM + Graph | N/A | matchlstm_graph.json |