This repository contains the code for our ACL 2018 paper:
Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting.
You can
- Look at the generated summaries and evaluate the ROUGE/METEOR scores
- Run decoding of the pretrained model
- Train your own model
If you use this code, please cite our paper:
@inproceedings{chen2018fast,
title={Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting},
author={Yen-Chun Chen and Mohit Bansal},
booktitle={Proceedings of ACL},
year={2018}
}
- Python 3 (tested on python 3.6)
- PyTorch 0.4.0
- with GPU and CUDA enabled installation (though the code is runnable on CPU, it would be way too slow)
- gensim
- cytoolz
- tensorboardX
- pyrouge (for evaluation)
You can use the python package manager of your choice (pip/conda) to install the dependencies. The code is tested on the Linux operating system.
Download the output summaries here.
To evaluate, you will need to download and setup the official ROUGE and METEOR packages.
We use pyrouge
(pip install pyrouge
to install)
to make the ROUGE XML files required by the official perl script.
You will also need the official ROUGE package.
(However, it seems that the original ROUGE website is down.
An alternative can be found
here.)
Please specify the path to your ROUGE package by setting the environment variable
export ROUGE=[path/to/rouge/directory]
.
For METEOR, we only need the JAR file meteor-1.5.jar
.
Please specify the file by setting the environment variable
export METEOR=[path/to/meteor/jar]
.
Run
python eval_acl.py --[rouge/meteor] --decode_dir=[path/to/decoded/files]
to get the ROUGE/METEOR scores reported in the paper.
Download the pretrained models here.
You will also need a preprocessed version of the CNN/DailyMail dataset.
Please follow the instructions
here
for downloading and preprocessing the CNN/DailyMail dataset.
After that, specify the path of data files by setting the environment variable
export DATA=[path/to/decompressed/data]
We provide 2 versions of pretrained models.
Using acl
you can reproduce the results reported in our paper.
Using new
you will get our latest result trained with a newer version of PyTorch library
which leads to slightly higher scores.
If using a Linux based system:
apt-get -y update && apt-get install libxml-parser-perl
Add the following environment variables:
export LANG=C.UTF-8
export PYTHONIOENCODING=latin-1
To decode, run
python decode_full_model.py --path=[path/to/save/decoded/files] --model_dir=[path/to/pretrained] --beam=[beam_size] [--test/--val]
Options:
- beam_size: number of hypothesis for (diverse) beam search. (use beam_size > 1 to enable reranking)
- beam_szie=1 to get greedy decoding results (rnn-ext + abs + RL)
- beam_size=5 is used in the paper for the +rerank model (rnn-ext + abs + RL + rerank)
- test/val: decode on test/validation dataset
If you want to evaluate on the generated output files, please follow the instructions in the above section to setup ROUGE/METEOR.
Next, make the reference files for evaluation:
python make_eval_references.py
git clone https://github.com/tagucci/pythonrouge.git
cd pythonrouge/RELEASE-1.5.5/data/
./WordNet-2.0-Exceptions/buildExeptionDB.pl ./WordNet-2.0-Exceptions ./smart_common_words.txt ./WordNet-2.0.exc.db
cp WordNet-2.0.exc.db pyrouge/tools/ROUGE-1.5.5/data/
export ROUGE=pyrouge/tools/ROUGE-1.5.5/data
and then run evaluation by:
python eval_full_model.py --[rouge/meteor] --decode_dir=[path/to/save/decoded/files]
You should get the following results
Validation set
Models | ROUGEs (R-1, R-2, R-L) | METEOR |
---|---|---|
acl | ||
rnn-ext + abs + RL | (41.01, 18.20, 38.57) | 21.10 |
+ rerank | (41.74, 18.39, 39.40) | 20.45 |
new | ||
rnn-ext + abs + RL | (41.23, 18.45, 38.71) | 21.14 |
+ rerank | (42.06, 18.80, 39.68) | 20.58 |
Test set
Models | ROUGEs (R-1, R-2, R-L) | METEOR |
---|---|---|
acl | ||
rnn-ext + abs + RL | (40.03, 17.61, 37.58) | 21.00 |
+ rerank | (40.88, 17.81, 38.53) | 20.38 |
new | ||
rnn-ext + abs + RL | (40.41, 17.92, 37.87) | 21.13 |
+ rerank | (41.20, 18.18, 38.79) | 20.56 |
NOTE: The original models in the paper are trained with pytorch 0.2.0 on python 2. After the acceptance of the paper, we figured it is better for the community if we release the code with latest libraries so that it becomes easier to build new models/techniques on top of our work. This results in a negligible difference w.r.t. our paper results when running the old pretrained model; and gives slightly better scores than our paper if running the new pretrained model.
Please follow the instructions
here
for downloading and preprocessing the CNN/DailyMail dataset.
After that, specify the path of data files by setting the environment variable
export DATA=[path/to/decompressed/data]
To re-train our best model:
- pretrained a word2vec word embedding
python train_word2vec.py --path=[path/to/word2vec]
- make the pseudo-labels
python make_extraction_labels.py
- train abstractor and extractor using ML objectives
python train_abstractor.py --path=[path/to/abstractor/model] --w2v=[path/to/word2vec/word2vec.128d.226k.bin]
python train_extractor_ml.py --path=[path/to/extractor/model] --w2v=[path/to/word2vec/word2vec.128d.226k.bin]
- train the full RL model
python train_full_rl.py --path=[path/to/save/model] --abs_dir=[path/to/abstractor/model] --ext_dir=[path/to/extractor/model]
After the training finishes you will be able to run the decoding and evaluation following the instructions in the previous section.
The above will use the best hyper-parameters we used in the paper as default. Please refer to the respective source code for options to set the hyper-parameters.