cylnlp/dialogsum

Rouge Score Scripts

Hannibal046 opened this issue · 5 comments

Hannibal046 commented

Hello,could you please tell which rouge scripts you use?thanks

cylnlp commented

Hi, following Gliwa et al. (2019), we used py-rouge for evaluation. You may find the script here: https://pypi.org/project/py-rouge/.

Hannibal046 commented

ok， thanks for the reply.
Did you insert \n between two sentence to calculate rouge score? Because in the link you post , official examples have, which will give a much higher rouge-L score。

cylnlp commented

Hi,
"\n"s are inserted as marks between utterances in dialogues, which are not inserted between sentences within an utternace.
For official examples have, do you mean \n between utterances?

Hannibal046 commented

in this link ,the hypothesis_1 and reference_1 are all appended \n after period, which is curcial for rouge-L.

cylnlp commented

Sorry for misunderstanding your question.
And the answer is no.
We directly feed the model-generated outputs and references into the py-rouge.