/llm-hallucination-survey

Reading list of hallucination in LLMs. Check out our new survey paper: "Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models"

llm-hallucination-survey

Version Stars Issues

Hallucination refers to the generated content that while seemingly plausible, deviates from user input (input-conflicting), previously generated context (context-conflicting), or factual knowledge (fact-conflicting).

LLM evaluation

This issue significantly undermines the reliability of LLMs in real-world scenarios.

📰News

😎 We have uploaded a comprehensive survey about the hallucination issue within the context of large language models, which discussed the evaluation, explanation, and mitigation. Check it out!

Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models

If you think our survey is helpful, please kindly cite our paper:

@article{zhang2023hallucination,
      title={Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models}, 
      author={Zhang, Yue and Li, Yafu and Cui, Leyang and Cai, Deng and Liu, Lemao and Fu, Tingchen and Huang, Xinting and Zhao, Enbo and Zhang, Yu and Chen, Yulong and Wang, Longyue and Luu, Anh Tuan and Bi, Wei and Shi, Freda and Shi, Shuming},
      journal={arXiv preprint arXiv:2309.01219},
      year={2023}
}

🚀Table of Content

🔍Evaluation of LLM Hallucination

Input-conflicting Hallucination

This kind of hallucination denotes the model response deviates from the user input, including task instruction and task input. This kind of hallucination has been widely studied in some traditional NLG tasks, such as:

  • Machine Translation:
    • Hallucinations in Neural Machine TranslationDownload Katherine Lee, Orhan Firat, Ashish Agarwal, Clara Fannjiang, David Sussillo [paper] 2018.9
    • Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translation Nuno M. Guerreiro, Elena Voita, André F.T. Martins [paper] 2022.8
    • Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even BetterDavid Dale, Elena Voita, Loïc Barrault, Marta R. Costa-jussà[paper] 2022.12
  • Data-to-text:
    • Controlling Hallucinations at Word Level in Data-to-Text Generation Clément Rebuffel, Marco Roberti, Laure Soulier, Geoffrey Scoutheeten, Rossella Cancelliere, Patrick Gallinari[paper] 2021.2
    • On Hallucination and Predictive Uncertainty in Conditional Language GenerationYijun Xiao, William Yang Wang[paper] 2021.3
  • Summarization:
    • On Faithfulness and Factuality in Abstractive Summarization Joshua Maynez, Shashi Narayan, Bernd Bohnet, Ryan McDonald[paper] 2020.5
    • Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization Meng Cao, Yue Dong, Jackie Chi Kit Cheung[paper] 2021.9
    • Summarization is (Almost) Dead Xiao Pu, Mingqi Gao, Xiaojun Wan[paper] 2023.9
    • Hallucination Reduction in Long Input Text Summarization Tohida Rehman, Ronit Mandal, Abhishek Agarwal, Debarshi Kumar Sanyal[paper] 2023.9
  • Dialogue:
    • Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding Nouha Dziri, Andrea Madotto, Osmar Zaiane, Avishek Joey Bose[paper] 2021.4
    • RHO: Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding Ziwei Ji, Zihan Liu, Nayeon Lee, Tiezheng Yu, Bryan Wilie, Min Zeng, Pascale Fung[paper] 2023.7
  • Question Answering:
    • Entity-Based Knowledge Conflicts in Question Answering Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, Sameer Singh[paper] 2021.9
    • Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answering Vaibhav Adlakha, Parishad BehnamGhader, Xing Han Lu, Nicholas Meade, Siva Reddy [paper] 2023.7

Context-conflicting Hallucination

This kind of hallucination means the generated content exhibits self-contradiction, i.e., conflicts with previously generated content. Here are some preliminary studies in this direction:

  1. Knowledge Enhanced Fine-Tuning for Better Handling Unseen Entities in Dialogue Generation Leyang Cui, Yu Wu, Shujie Liu, Yue Zhang[paper] 2021.9

  2. A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation Tianyu Liu, Yizhe Zhang, Chris Brockett, Yi Mao, Zhifang Sui, Weizhu Chen, Bill Dolan[paper] 2022.5 (not only limited to context-conflicting type)

  3. Large Language Models Can Be Easily Distracted by Irrelevant Context Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H. Chi, Nathanael Schärli, Denny Zhou[paper] 2023.2

  4. HistAlign: Improving Context Dependency in Language Generation by Aligning with History David Wan, Shiyue Zhang, Mohit Bansal[paper] 2023.5

  5. Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation Niels Mündler, Jingxuan He, Slobodan Jenko, Martin Vechev [paper] 2023.5

Fact-conflicting Hallucination

This kind of hallucination means the generated content conflicts with established facts. This kind of hallucination is challenging and important for practical applications of LLMs, so it has been widely studied in recent work.

  1. TruthfulQA: Measuring How Models Mimic Human Falsehoods Stephanie Lin, Jacob Hilton, Owain Evans [paper] 2022.5

  2. A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation Tianyu Liu, Yizhe Zhang, Chris Brockett, Yi Mao, Zhifang Sui, Weizhu Chen, Bill Dolan [paper] 2022.5

  3. A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, Pascale Fung [paper] 2023.2

  4. HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, Ji-Rong Wen [paper] 2023.5

  5. Automatic Evaluation of Attribution by Large Language Models Xiang Yue, Boshi Wang, Kai Zhang, Ziru Chen, Yu Su, Huan Sun [paper] 2023.5

  6. Adaptive Chameleon or Stubborn Sloth: Unraveling the Behavior of Large Language Models in Knowledge Clashes Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, Yu Su [paper] 2023.5

  7. LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond Philippe Laban, Wojciech Kryściński, Divyansh Agarwal, Alexander R. Fabbri, Caiming Xiong, Shafiq Joty, Chien-Sheng Wu [paper] 2023.5

  8. Evaluating the Factual Consistency of Large Language Models Through News Summarization Derek Tam, Anisha Mascarenhas, Shiyue Zhang, Sarah Kwan, Mohit Bansal, Colin Raffel [paper] 2023.5

  9. Methods for Measuring, Updating, and Visualizing Factual Beliefs in Language Models Peter Hase, Mona Diab, Asli Celikyilmaz, Xian Li, Zornitsa Kozareva, Veselin Stoyanov, Mohit Bansal, Srinivasan Iyer [paper] 2023.5

  10. How Language Model Hallucinations Can Snowball Muru Zhang, Ofir Press, William Merrill, Alisa Liu, Noah A. Smith [paper] 2023.5

  11. Evaluating Factual Consistency of Texts with Semantic Role Labeling Jing Fan, Dennis Aumiller, Michael Gertz [paper] 2023.5

  12. FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, Hannaneh Hajishirzi [paper] 2023.5

  13. Measuring and Modifying Factual Knowledge in Large Language Models Pouya Pezeshkpour [paper] 2023.6

  14. KoLA: Carefully Benchmarking World Knowledge of Large Language Models Jifan Yu, Xiaozhi Wang, Shangqing Tu, Shulin Cao, Daniel Zhang-Li, Xin Lv, Hao Peng, Zijun Yao, Xiaohan Zhang, Hanming Li, Chunyang Li, Zheyuan Zhang, Yushi Bai, Yantao Liu, Amy Xin, Nianyi Lin, Kaifeng Yun, Linlu Gong, Jianhui Chen, Zhili Wu, Yunjia Qi, Weikai Li, Yong Guan, Kaisheng Zeng, Ji Qi, Hailong Jin, Jinxin Liu, Yu Gu, Yuan Yao, Ning Ding, Lei Hou, Zhiyuan Liu, Bin Xu, Jie Tang, Juanzi Li [paper] 2023.6

  15. Generating Benchmarks for Factuality Evaluation of Language Models Dor Muhlgay, Ori Ram, Inbal Magar, Yoav Levine, Nir Ratner, Yonatan Belinkov, Omri Abend, Kevin Leyton-Brown, Amnon Shashua, Yoav Shoham [paper] 2023.7

  16. Fact-Checking of AI-Generated Reports Razi Mahmood, Ge Wang, Mannudeep Kalra, Pingkun Yan [paper] 2023.7

  17. Med-HALT: Medical Domain Hallucination Test for Large Language Models Logesh Kumar Umapathi, Ankit Pal, Malaikannan Sankarasubbu [paper] 2023.7

  18. Large Language Models on Wikipedia-Style Survey Generation: an Evaluation in NLP Concepts

    Fan Gao, Hang Jiang, Moritz Blum, Jinghui Lu, Yuang Jiang, Irene Li [paper] 2023.8

  19. ChatGPT Hallucinates when Attributing Answers Guido Zuccon, Bevan Koopman, Razia Shaik [paper] 2023.9

  20. BAMBOO: A Comprehensive Benchmark for Evaluating Long Text Modeling Capacities of Large Language Models Zican Dong, Tianyi Tang, Junyi Li, Wayne Xin Zhao, Ji-Rong Wen [paper] 2023.9

  21. KLoB: a Benchmark for Assessing Knowledge Locating Methods in Language Models Yiming Ju, Zheng Zhang [paper] 2023.9

  22. AutoHall: Automated Hallucination Dataset Generation for Large Language Models Zouying Cao, Yifei Yang, Hai Zhao [paper] 2023.10

  23. FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation Tu Vu, Mohit Iyyer, Xuezhi Wang, Noah Constant, Jerry Wei, Jason Wei, Chris Tar, Yun-Hsuan Sung, Denny Zhou, Quoc Le, Thang Luong [paper] 2023.10

  24. Evaluating Hallucinations in Chinese Large Language Models Qinyuan Cheng, Tianxiang Sun, Wenwei Zhang, Siyin Wang, Xiangyang Liu, Mozhi Zhang, Junliang He, Mianqiu Huang, Zhangyue Yin, Kai Chen, Xipeng Qiu [paper] 2023.10

  25. FELM: Benchmarking Factuality Evaluation of Large Language Models Shiqi Chen, Yiran Zhao, Jinghan Zhang, I-Chun Chern, Siyang Gao, Pengfei Liu, Junxian He [paper] 2023.10

🚨Source of LLM Hallucination

There is also a line of works that try to explain the hallucination with LLMs.

  1. How Pre-trained Language Models Capture Factual Knowledge? A Causal-Inspired Analysis Shaobo Li, Xiaoguang Li, Lifeng Shang, Zhenhua Dong, Chengjie Sun, Bingquan Liu, Zhenzhou Ji, Xin Jiang, Qun Liu [paper] 2022.3

  2. On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models? Nouha Dziri, Sivan Milton, Mo Yu, Osmar Zaiane, Siva Reddy [paper] 2022.4

  3. Towards Tracing Factual Knowledge in Language Models Back to the Training Data Ekin Akyürek, Tolga Bolukbasi, Frederick Liu, Binbin Xiong, Ian Tenney, Jacob Andreas, Kelvin Guu [paper] 2022.5

  4. Language Models (Mostly) Know What They Know Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, Deep Ganguli, Danny Hernandez, Josh Jacobson, Jackson Kernion, Shauna Kravec, Liane Lovitt, Kamal Ndousse, Catherine Olsson, Sam Ringer, Dario Amodei, Tom Brown, Jack Clark, Nicholas Joseph, Ben Mann, Sam McCandlish, Chris Olah, Jared Kaplan [paper] 2022.7

  5. Discovering Language Model Behaviors with Model-Written Evaluations Ethan Perez, Sam Ringer, Kamilė Lukošiūtė, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, Andy Jones, Anna Chen, Ben Mann, Brian Israel, Bryan Seethor, Cameron McKinnon, Christopher Olah, Da Yan, Daniela Amodei, Dario Amodei, Dawn Drain, Dustin Li, Eli Tran-Johnson, Guro Khundadze, Jackson Kernion, James Landis, Jamie Kerr, Jared Mueller, Jeeyoon Hyun, Joshua Landau, Kamal Ndousse, Landon Goldberg, Liane Lovitt, Martin Lucas, Michael Sellitto, Miranda Zhang, Neerav Kingsland, Nelson Elhage, Nicholas Joseph, Noemí Mercado, Nova DasSarma, Oliver Rausch, Robin Larson, Sam McCandlish, Scott Johnston, Shauna Kravec, Sheer El Showk, Tamera Lanham, Timothy Telleen-Lawton, Tom Brown, Tom Henighan, Tristan Hume, Yuntao Bai, Zac Hatfield-Dodds, Jack Clark, Samuel R. Bowman, Amanda Askell, Roger Grosse, Danny Hernandez, Deep Ganguli, Evan Hubinger, Nicholas Schiefer, Jared Kaplan [paper] 2022.12

  6. Why Does ChatGPT Fall Short in Providing Truthful Answers? Shen Zheng, Jie Huang, Kevin Chen-Chuan Chang [paper] 2023.4

  7. Do Large Language Models Know What They Don't Know? Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, Xuanjing Huang [paper] 2023.5

  8. Sources of Hallucination by Large Language Models on Inference Tasks

    Nick McKenna, Tianyi Li, Liang Cheng, Mohammad Javad Hosseini, Mark Johnson, Mark Steedman [paper] 2023.5

  9. Enabling Large Language Models to Generate Text with Citations Tianyu Gao, Howard Yen, Jiatong Yu, Danqi Chen [paper] 2023.5

  10. Overthinking the Truth: Understanding how Language Models Process False Demonstrations Danny Halawi, Jean-Stanislas Denain, Jacob Steinhardt [paper] 2023.7

  11. Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation Ruiyang Ren, Yuhao Wang, Yingqi Qu, Wayne Xin Zhao, Jing Liu, Hao Tian, Hua Wu, Ji-Rong Wen, Haifeng Wang [paper] 2023.7

  12. Head-to-Tail: How Knowledgeable are Large Language Models (LLM)? A.K.A. Will LLMs Replace Knowledge Graphs? Kai Sun, Yifan Ethan Xu, Hanwen Zha, Yue Liu, Xin Luna Dong [paper] 2023.8

  13. Simple synthetic data reduces sycophancy in large language models Jerry Wei, Da Huang, Yifeng Lu, Denny Zhou, Quoc V. Le [paper] 2023.8

  14. Do PLMs Know and Understand Ontological Knowledge? Weiqi Wu, Chengyue Jiang, Yong Jiang, Pengjun Xie, Kewei Tu [paper] 2023.9

  15. Exploring the Relationship between LLM Hallucinations and Prompt Linguistic Nuances: Readability, Formality, and Concreteness Vipula Rawte, Prachi Priya, S.M Towhidul Islam Tonmoy, S M Mehedi Zaman, Amit Sheth, Amitava Das [paper] 2023.9

  16. LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples Jia-Yu Yao, Kun-Peng Ning, Zhen-Hui Liu, Mu-Nan Ning, Li Yuan [paper] 2023.10

🛠Mitigation of LLM Hallucination

Numerous recent work tries to mitigate hallucination in LLMs. These methods can be applied at different stages of LLM life cycle.

Mitigation During Pretraining

One main mitigation method during pretraining is (automatically) curating training data. Here are some papers using this method:

  1. Factuality Enhanced Language Models for Open-Ended Text Generation Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Pascale Fung, Mohammad Shoeybi, Bryan Catanzaro [paper] 2022.6
  2. The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, Julien Launay [paper] 2023.7
  3. Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, Thomas Scialom [paper] 2023.7
  4. Textbooks Are All You Need II: phi-1.5 technical report Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, Yin Tat Lee [paper] 2023.9

Mitigation During SFT

Mitigating hallucination during SFT can involve curating SFT data, such as:

  1. LIMA: Less Is More for Alignment Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, Omer Levy [paper] 2023.5
  2. AlpaGasus: Training A Better Alpaca with Fewer Data Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, Hongxia Jin [paper] 2023.7
  3. Instruction Mining: High-Quality Instruction Data Selection for Large Language Models Yihan Cao, Yanbin Kang, Lichao Sun [paper] 2023.7
  4. Halo: Estimation and Reduction of Hallucinations in Open-Source Weak Large Language Models Mohamed Elaraby, Mengyin Lu, Jacob Dunn, Xueying Zhang, Yu Wang, Shizhu Liu [paper] 2023.8

Some researchers claim that the behavior cloning phenomenon in SFT can induce hallucinations. So some works try to mitigate hallucinations via honesty-oriented SFT.

  1. MOSS: Training Conversational Language Models from Synthetic Data Tianxiang Sun and Xiaotian Zhang and Zhengfu He and Peng Li and Qinyuan Cheng and Hang Yan and Xiangyang Liu and Yunfan Shao and Qiong Tang and Xingjian Zhao and Ke Chen and Yining Zheng and Zhejian Zhou and Ruixiao Li and Jun Zhan and Yunhua Zhou and Linyang Li and Xiaogui Yang and Lingling Wu and Zhangyue Yin and Xuanjing Huang and Xipeng Qiu [repo] 2023

Mitigation During RLHF

  1. Training language models to follow instructions with human feedback Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, Ryan Lowe [paper] 2022.3
  2. GPT-4 Technical Report OpenAI [paper] 2023.3
  3. Let's Verify Step by Step Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, Karl Cobbe [paper] 2023.5
  4. Reinforcement learning from human feedback: Progress and challenges John Schulman [talk] 2023.5
  5. Fine-Grained Human Feedback Gives Better Rewards for Language Model Training Zeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri, Alane Suhr, Prithviraj Ammanabrolu, Noah A. Smith, Mari Ostendorf, Hannaneh Hajishirzi [paper] 2023.6
  6. Aligning Large Multimodal Models with Factually Augmented RLHF Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Gui, Yu-Xiong Wang, Yiming Yang, Kurt Keutzer, Trevor Darrell [paper] 2023.9
  7. Human Feedback is not Gold Standard Tom Hosking, Phil Blunsom, Max Bartolo [paper] 2023.9
  8. Tool-Augmented Reward Modeling Lei Li, Yekun Chai, Shuohuan Wang, Yu Sun, Hao Tian, Ningyu Zhang, Hua Wu [paper] 2023.10

Mitigation During Inference

Designing Decode Strategy

  1. Factuality Enhanced Language Models for Open-Ended Text Generation Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Pascale Fung, Mohammad Shoeybi, Bryan Catanzaro [paper] 2022.6

  2. When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, Hannaneh Hajishirzi [paper] 2022.10

  3. Trusting Your Evidence: Hallucinate Less with Context-aware Decoding Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, Scott Wen-tau Yih [paper] 2023.5

  4. Inference-Time Intervention: Eliciting Truthful Answers from a Language Model Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, Martin Wattenberg [paper] 2023.6

  5. DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, Pengcheng He [paper] 2023.9

  6. Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decoding Rico Sennrich, Jannis Vamvas, Alireza Mohammadshahi [paper] 2023.9

  7. Chain-of-Verification Reduces Hallucination in Large Language Models Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, Jason Weston [paper] 2023.9

Resorting to External Knowledge

  1. RARR: Researching and Revising What Language Models Say, Using Language Models Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Y. Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, Kelvin Guu [paper] 2022.10

  2. Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, Jianfeng Gao [paper] 2023.2

  3. GeneGPT: Augmenting Large Language Models with Domain Tools for Improved Access to Biomedical Information Qiao Jin, Yifan Yang, Qingyu Chen, Zhiyong Lu [paper] 2023.4

  4. Zero-shot Faithful Factual Error Correction Kung-Hsiang Huang, Hou Pong Chan, Heng Ji [paper] 2023.5

  5. CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, Weizhu Chen [paper] 2023.5

  6. PURR: Efficiently Editing Language Model Hallucinations by Denoising Language Model Corruptions Anthony Chen, Panupong Pasupat, Sameer Singh, Hongrae Lee, Kelvin Guu [paper] 2023.5

  7. Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Framework Ruochen Zhao, Xingxuan Li, Shafiq Joty, Chengwei Qin, Lidong Bing [paper] 2023.5

  8. Self-Checker: Plug-and-Play Modules for Fact-Checking with Large Language Models
    Miaoran Li, Baolin Peng, Zhu Zhang [paper] 2023.5

  9. Augmented Large Language Models with Parametric Knowledge Guiding Ziyang Luo, Can Xu, Pu Zhao, Xiubo Geng, Chongyang Tao, Jing Ma, Qingwei Lin, Daxin Jiang [paper] 2023.5

  10. FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios I-Chun Chern, Steffi Chern, Shiqi Chen, Weizhe Yuan, Kehua Feng, Chunting Zhou, Junxian He, Graham Neubig, Pengfei Liu [paper] 2023.7

  11. Knowledge Solver: Teaching LLMs to Search for Domain Knowledge from Knowledge Graphs Chao Feng, Xinyu Zhang, Zichu Fei [paper] 2023.9

  12. "Merge Conflicts!" Exploring the Impacts of External Distractors to Parametric Knowledge Graphs Cheng Qian, Xinran Zhao, Sherry Tongshuang Wu [paper] 2023.9

  13. BTR: Binary Token Representations for Efficient Retrieval Augmented Language Models Qingqing Cao, Sewon Min, Yizhong Wang, Hannaneh Hajishirzi [paper] 2023.10

  14. FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation Tu Vu, Mohit Iyyer, Xuezhi Wang, Noah Constant, Jerry Wei, Jason Wei, Chris Tar, Yun-Hsuan Sung, Denny Zhou, Quoc Le, Thang Luong [paper] 2023.10

Exploiting Uncertainty

  1. SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models Potsawee Manakul, Adian Liusie, Mark J. F. Gales [paper] 2023.3

  2. Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation Niels Mündler, Jingxuan He, Slobodan Jenko, Martin Vechev [paper] 2023.5

  3. Do Language Models Know When They're Hallucinating References? Ayush Agrawal, Lester Mackey, Adam Tauman Kalai [paper] 2023.5

  4. LLM Calibration and Automatic Hallucination Detection via Pareto Optimal Self-supervision Theodore Zhao, Mu Wei, J. Samuel Preston, Hoifung Poon [paper] 2023.6

  5. A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation Neeraj Varshney, Wenlin Yao, Hongming Zhang, Jianshu Chen, Dong Yu [paper] 2023.7

  6. Zero-Resource Hallucination Prevention for Large Language Models Junyu Luo, Cao Xiao, Fenglong Ma [paper] 2023.9

  7. Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models Mert Yuksekgonul, Varun Chandrasekaran, Erik Jones, Suriya Gunasekar, Ranjita Naik, Hamid Palangi, Ece Kamar, Besmira Nushi [paper] 2023.9

Multi-agent Interaction

  1. Improving Factuality and Reasoning in Language Models through Multiagent Debate Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, Igor Mordatch [paper] 2023.5

  2. LM vs LM: Detecting Factual Errors via Cross Examination Roi Cohen, May Hamri, Mor Geva, Amir Globerson [paper] 2023.5

  3. Unleashing Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, Heng Ji [paper] 2023.7

Human-in-the-loop

  1. Mitigating Language Model Hallucination with Interactive Question-Knowledge Alignment Shuo Zhang, Liangming Pan, Junzhou Zhao, William Yang Wang [paper] 2023.5

Analyzing Internal Model States

  1. The Internal State of an LLM Knows When its Lying Amos Azaria, Tom Mitchell [paper] 2023.4

  2. Do Language Models Know When They're Hallucinating References? Ayush Agrawal, Lester Mackey, Adam Tauman Kalai [paper] 2023.5

  3. Inference-Time Intervention: Eliciting Truthful Answers from a Language Model Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, Martin Wattenberg [paper] 2023.6

  4. Knowledge Sanitization of Large Language Models Yoichi Ishibashi, Hidetoshi Shimodaira [paper] 2023.9

📲Contact

We warmly welcome any kinds of useful suggestions or contributions. Feel free to drop us an issue or contact Hill with this e-mail.