This PyTorch package implements the Multi-Task Stochastic Answer Network (MT-SAN) for Machine Reading Comprehension, as described in:
Yichong Xu, Xiaodong Liu, Yelong Shen, Jingjing Liu and Jianfeng Gao
Multi-task Learning with Sample Re-weighting for Machine Reading Comprehension
North American Chapter of the Association for Computational Linguistics (NAACL), 2019
arXiv version
Please cite the above paper if you use this code.
We report single-model results produced by this package as follows.
| Dataset | EM/F1 |
|---|---|
SQuAD v1.1 (Rajpurkar et al., 2016) |
81.6/88.2 (development set) |
NewsQA (Trischler et al., 2016) |
59.9/72.6 (vs. 46.5/69.4 Human Performance) |
-
python3.6
-
install requirements
> pip install -r requirements.txt
-
pull docker: We can use the same docker as mt-dnn
> docker pull allenlao/pytorch-mt-dnn:v0.1 -
run docker
> docker run -it --rm --runtime nvidia allenlao/pytorch-mt-dnn:v0.1 bash
Please refer to the following link if you first use docker: https://docs.docker.com/
- prepare data
./prepare_data.sh
- train a model: See example codes in run.sh
You can download pretrained models with the best SQuAD performance here.
The code is developed based on the original SAN code: https://github.com/kevinduh/san_mrc
Related: MT-DNN