/Question-Answering-based-on-SQuAD

Question Answering System using BiDAF Model on SQuAD v2.0

Primary LanguagePython

Question Answering with SQuAD using BiDAF model

Hits

Implemented a Bidirectional Attention Flow neural network as a baseline, improving Chris Chute's model implementation, adding word-character inputs as described in the original paper and improving GauthierDmns' code.

You can reproduce the work following the Set-Up section, and potentially (recommended!) train the model on a single GPU setting the cuda variable in config.py to True.

SQuAD

Question-answer pairs for a sample passage in the SQuAD dataset. Each of the answers is a segment of text from the passage.

Model Architecture

BiDAF Architecture Source: BiDAF paper

Requirements

  • Python 3.6

Code Organization

├── config.py          <- Configuration file with data directories and hyperparamters to train the model
├── data_loader.py     <- Define an iterator who collects batches of data to train the model
├── eval.py            <- Evaluate the model on a new pair of (context, question)
├── layers.py          <- Define the various layers to be used by the main BiDAF model
├── make_dataset.py    <- Download the SquAD dataset and pre-process the data for training
├── model.py.          <- Define the BiDAF model architecture
├── requirements.txt   <- Required Python libraries to build the project
├── test.py            <- Test the performance of a trained model on the DEV dataset
├── train.py           <- Train a model using the TRAIN dataset only
├── utils.py           <- Group a bunch of useful functions to process the data

Results

Loss and Metrics

Exact-Match and F1 Score on Validation set after training:

EM F1
0.64 0.75

Set-Up

  • Clone the repository
  • Create a directory for your experiments, logs and model weights: mkdir output
  • Download GloVE word vectors: https://nlp.stanford.edu/projects/glove/
  • Modify the config.py file to set up the paths where your GloVE, SquAD and models will be located
  • Create a Python virtual environment, source to it: mkvirualenv qa-env ; workon qa-env if you use virtualenvwrapper
  • Install the dependencies: pip install -r requirements.txt ; python -m spacy download en
  • Run python make_dataset.py to download SquAD dataset and pre-process the data
  • Run python train.py to train the model with hyper-parameters found in config.py
  • Run python test.py to test the model EM and F1 scores on Dev examples
  • Play with eval.py to answer your own questions! :)

Next Steps

  • set up a variable to choose between training the model with word only VS word + characters
  • collect the moving average of the weights during training and use them during testing
  • add the ability to train the model on multiple GPUs, and offer half-precision training to speed-up the training
  • improve this baseline using pre-training encoding such as BERT, and/or set-up a multi-task learning pipeline to jointly learn to answer questions together with another closely related NLP task.

Useful Articles

Useful Links

Problems that can be