Representation Learning for Answer Selection with LSTM-Based Importance Weighting

Paper: http://www.aclweb.org/anthology/W17-6935

The source code in this project allows you to replicate our results reported in the associated paper. The project can also be used as a starting point for new experiments.

Please use the following citation:

@InProceedings{rueckle:2017:IWCS,
  title = {Representation Learning for Answer Selection with LSTM-Based Importance Weighting},
  author = {R{\"u}ckl{\'e}, Andreas and Gurevych, Iryna},
  publisher = {Association for Computational Linguistics},
  booktitle = {Proceedings of the 12th International Conference on Computational Semantics (IWCS 2017)},
  month = sep,
  year = {2017},
  location = {Montpellier, France}
}

Contact person: Andreas Rücklé

https://www.ukp.tu-darmstadt.de/

https://www.tu-darmstadt.de/

Don't hesitate to send us an e-mail or report an issue, if something is broken (and it shouldn't be) or if you have further questions.

This repository contains experimental software and is published for the sole purpose of giving additional background details on the respective publication.

Experimental Framework

In the following, we briefly describe the different components that are included in this project and list the steps required to run LW experiments.

Project Structure

The project includes the following files and folder:

/configs: A folder that contains configuration files that can be used to run the experiments of the associated paper
/experiment: Contains the source code of all models, the training procedure and the evaluation
/scripts: Contains additional scripts for hyperparameter optimization
/tests: Unit tests
default_config.yaml: The standard configuration fallback
requirements.txt: Lists the required python packages. For information on the required tensorflow versions, please see the notes below
run_experiment.py: The main entry point for each experiment

Configuration Files

Each experiment requires a dedicated YAML configuration file. It defines which components should be loaded for execution. Each experiment can define four components:

Data: Defines which data loading module should be used. We included modules for InsuranceQA version 1 and 2 as well as WikiQA
Model: The model which should be used for training and evaluation
Training: The training module implements the training procedure
Evaluation (optional): Performs the final evaluation

Each module can be configured in separate sections of the configuration file. The configurations used in our experiments can be found in the folder /configs.

Running Experiments

We used Python 2.7.6 with TensorFlow 0.11.0rc0 on a Nvidia Tesla K40 GPU for all InsuranceQA experiments, and Python 3.5.3 with TensorFlow 0.10.0 on a Nvidia K20x GPU for all WikiQA experiments.

To run the experiments:

Install all requirements listed in requirements.txt (> pip install requirements.txt)
Install TensorFlow
Download InsuranceQA (https://github.com/shuzi/insuranceQA)
Download GloVe Embeddings and unzip them http://nlp.stanford.edu/data/glove.6B.zip
Change the desired configuration file in /configs and replace all placeholders
run python run_experiment <path-to>/config.yaml

Visualization and End2End QA

For analysis and to visualize the attention weights we used our framework described in the demo paper "End-to-End Non-Factoid Question Answering with an Interactive Visualization of Neural Attention Weights" (Rücklé and Gurevych, ACL 2017). The source code is available here.