/LLM4IR-Survey

This is the repo for the survey of LLM4IR.

MIT LicenseMIT

LLM4IR-Survey

This is the collection of papers related to large language models for information retrieval. These papers are organized according to our survey paper Large Language Models for Information Retrieval: A Survey.

Feel free to contact us if you find a mistake or have any advice. Email: yutaozhu94@gmail.com and dou@ruc.edu.cn.

Please kindly cite our paper if helps your research:

@article{LLM4IRSurvey,
    author={Yutao Zhu and
            Huaying Yuan and
            Shuting Wang and
            Jiongnan Liu and
            Wenhan Liu and
            Chenlong Deng and
            Zhicheng Dou and
            Ji-Rong Wen},
    title={Large Language Models for Information Retrieval: A Survey},
    journal={CoRR},
    volume={abs/2308.07107},
    year={2023},
    url={https://arxiv.org/abs/2308.07107},
    eprinttype={arXiv},
    eprint={2306.07401}
}

Table of Content

Paper List

Query Rewriter

Prompting Methods

  1. Query2doc: Query Expansion with Large Language Models, Wang et al., arXiv 2023. [Paper]
  2. Generative and Pseudo-Relevant Feedback for Sparse, Dense and Learned Sparse Retrieval, Mackie et al., arXiv 2023. [Paper]
  3. Generative Relevance Feedback with Large Language Models, Mackie et al., SIGIR 2023 (short paper). [Paper]
  4. GRM: Generative Relevance Modeling Using Relevance-Aware Sample Estimation for Document Retrieval, Mackie et al., arXiv 2023. [Paper]
  5. Large Language Models Know Your Contextual Search Intent: A Prompting Framework for Conversational Search, Mao et al., arXiv 2023. [Paper]
  6. Precise Zero-Shot Dense Retrieval without Relevance Labels, Gao et al., ACL 2023. [Paper]
  7. Query Expansion by Prompting Large Language Models, Jagerman et al., arXiv 2023. [Paper]
  8. Large Language Models are Strong Zero-Shot Retriever, Shen et al., arXiv 2023. [Paper]
  9. Enhancing Conversational Search: Large Language Model-Aided Informative Query Rewriting, Ye et al., EMNLP 2023 Findings. [Paper]

Fine-tuning Methods

  1. QUILL: Query Intent with Large Language Models using Retrieval Augmentation and Multi-stage Distillation, Srinivasan et al., EMNLP 2022 (Industry). [Paper] (This paper explore fine-tuning methods in baseline experiments.)

Knowledge Distillation Methods

  1. QUILL: Query Intent with Large Language Models using Retrieval Augmentation and Multi-stage Distillation, Srinivasan et al., EMNLP 2022 (Industry). [Paper]
  2. Knowledge Refinement via Interaction Between Search Engines and Large Language Models, Feng et al., arXiv 2023. [Paper]
  3. Query Rewriting for Retrieval-Augmented Large Language Models, Ma et al., arXiv 2023. [Paper]

Retriever

Leveraging LLMs to Generate Search Data

  1. InPars: Data Augmentation for Information Retrieval using Large Language Models, Bonifacio et al., arXiv 2022. [Paper]
  2. InPars-v2: Large Language Models as Efficient Dataset Generators for Information Retrieval, Jeronymo et al., arXiv 2023. [Paper]
  3. Promptagator: Few-shot Dense Retrieval From 8 Examples, Dai et al., ICLR 2023. [Paper]
  4. AugTriever: Unsupervised Dense Retrieval by Scalable Data Augmentation, Meng et al., arXiv 2023. [Paper]
  5. UDAPDR: Unsupervised Domain Adaptation via LLM Prompting and Distillation of Rerankers, Saad-Falco et al., arXiv 2023. [Paper]
  6. Soft Prompt Tuning for Augmenting Dense Retrieval with Large Language Models, Peng et al., arXiv 2023. [Paper]
  7. Questions Are All You Need to Train a Dense Passage Retriever, Sachan et al., ACL 2023. [Paper]
  8. Beyond Factuality: A Comprehensive Evaluation of Large Language Models as Knowledge Generators, Chen et al., EMNLP 2023. [Paper]

Employing LLMs to Enhance Model Architecture

  1. Text and Code Embeddings by Contrastive Pre-Training, Neelakantan et al., arXiv 2022. [Paper]
  2. Large Dual Encoders Are Generalizable Retrievers, Ni et al., ACL 2022. [Paper]
  3. Task-aware Retrieval with Instructions, Asai et al., ACL 2023 (Findings). [Paper]
  4. Transformer memory as a differentiable search index, Tay et al., NeurIPS 2022. [Paper]
  5. Large Language Models are Built-in Autoregressive Search Engines, Ziems et al., ACL 2023 (Findings). [Paper]

Reranker

Fine-tuning LLMs for Reranking

  1. Document Ranking with a Pretrained Sequence-to-Sequence Model, Nogueira et al., EMNLP 2020 (Findings). [Paper]
  2. Text-to-Text Multi-view Learning for Passage Re-ranking, Ju et al., SIGIR 2021 (Short Paper). [Paper]
  3. The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Models, Pradeep et al., arXiv 2021. [Paper]
  4. RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses, Zhuang et al., SIGIR 2023 (Short Paper). [Paper]

Prompting LLMs for Reranking

  1. Holistic Evaluation of Language Models, Liang et al., arXiv 2022. [Paper]
  2. Improving Passage Retrieval with Zero-Shot Question Generation, Sachan et al., EMNLP 2022. [Paper]
  3. Discrete Prompt Optimization via Constrained Generation for Zero-shot Re-ranker, Cho et al., ACL 2023 (Findings). [Paper]
  4. Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agent, Sun et al., arXiv 2023. [Paper]
  5. Zero-Shot Listwise Document Reranking with a Large Language Model, Ma et al., arXiv 2023. [Paper]
  6. Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting, Qin et al., arXiv 2023. [Paper]

Utilizing LLMs for Training Data Augmentation

  1. ExaRanker: Explanation-Augmented Neural Ranker, Ferraretto et al., SIGIR 2023 (Short Paper). [Paper]
  2. InPars-Light: Cost-Effective Unsupervised Training of Efficient Rankers, Boytsov et al., arXiv 2023. [Paper]
  3. Generating Synthetic Documents for Cross-Encoder Re-Rankers, Askari et al., arXiv 2023. [Paper]
  4. Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agent, Sun et al., arXiv 2023. [Paper]

Reader

Passive Reader

  1. REALM: Retrieval-Augmented Language Model Pre-Training, Guu et al., arXiv 2020. [Paper]
  2. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, Lewis et al., NeurIPS 2020. [Paper]
  3. REPLUG: Retrieval-Augmented Black-Box Language Models, Shi et al., arXiv 2023. [Paper]
  4. Atlas: Few-shot Learning with Retrieval Augmented Language Models, Izacard et al., arXiv 2022. [Paper]
  5. Internet-augmented Language Models through Few-shot Prompting for Open-domain Question Answering, Lazaridou et al., arXiv 2022. [Paper]
  6. Rethinking with Retrieval: Faithful Large Language Model Inference, He et al., arXiv 2023. [Paper]
  7. RETA-LLM: A Retrieval-Augmented Large Language Model Toolkit, Liu et al., arXiv 2023. [Paper]
  8. In-Context Retrieval-Augmented Language Models, Ram et al., arXiv 2023. [Paper]
  9. Improving Language Models by Retrieving from Trillions of Tokens, Borgeaud et al., ICML 2022. [Paper]
  10. Interleaving Retrieval with Chain-of-thought Reasoning for Knowledge-intensive Multi-step Questions, Trivedi et al., ACL 2023, [Paper]
  11. Active Retrieval Augmented Generation, Jiang et al., arXiv 2023. [Paper]

Active Reader

  1. Measuring and Narrowing the Compositionality Gap in Language Models, Press et al., arXiv 2022, [Paper]
  2. DEMONSTRATE–SEARCH–PREDICT: Composing Retrieval and Language Models for Knowledge-intensive NLP, Khattab et al., arXiv 2022, [Paper]
  3. Answering Questions by Meta-Reasoning over Multiple Chains of Thought, Yoran et al., arXiv 2023, [Paper]
  4. WebGPT: Browser-assisted Question-answering with Human Feedback, Nakano et al., arXiv 2021. [Paper]
  5. WebCPM: Interactive Web Search for Chinese Long-form Question Answering, Qin et al., ACL 2023. [Paper]

Other Resources

  1. ACL 2023 Tutorial: Retrieval-based Language Models and Applications, Asai et al., ACL 2023. [Link]
  2. A Survey of Large Language Models, Zhao et al., arXiv 2023. [Paper]
  3. Information Retrieval Meets Large Language Models: A Strategic Report from Chinese IR Community, Ai et al., arXiv 2023. [Paper]