FFCI is a framework for fine-grained summarization evaluation that comprises four elements: Faithfulness (degree of factual consistency with the source), Focus (precision of summary content relative to the reference), Coverage (recall of summary content relative to the reference), and Inter-sentential coherence (document fluency between adjacent sentences).
Fajri Koto, Timothy Baldwin, and Jey Han Lau. FFCI: A Framework for Interpretable Automatic Evaluation of Summarization. Under review for the Journal of Artificial Intelligence Research (JAIR 2021).
You can find the data used in our experiments in folder Data.