/long-doc-summ

Comparing long document summarizaiton performance of LLMs

Primary LanguageJupyter Notebook

Long Document Summarization

This is a research project in long document summatization conducted in summer 2023

Tasks:

  • Prepare data
    • Booksum ✅
    • SummScreen in progress
    • Real books! coming in future
  • Eval
    • Automated
      • BERTScore ✅
      • R1, R2, R-L ✅
      • METEOR ✅
    • Coherence
      • Snac in progress
    • Faithfulness
      • QAFactEval
        • work on running it on compute cluster
        • swap out default question generator for gpt
    • Coverage
  • Summary generation