The approach in this part mainly involves fine-tuning LLMs using domain-specific data, thereby resulting in numerous vertical domain LLMs. The generation of their dataset can be sourced from domain-specific knowledge graphs or online data, and so on. Here, we mainly list some open-source large models in the medical field.
- BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining, Briefings in Bioinformatics, 2022
- DoctorGLM: Fine-tuning your Chinese Doctor is not a Herculean Task, Arxiv, 2023
- ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge, Arxiv, 2023
- PMC-LLaMA: Towards Building Open-source Language Models for Medicine, Arxiv, 2023
- HuatuoGPT, Towards Taming Language Models To Be a Doctor, Arxiv, 2023
The approach in this part is mainly to integrate the domain knowledge base into LLMs, usually involving the graph related algorithms and the retrieval way from the knowledge base.
- DecAF: Joint Decoding of Answers and Logical Forms for Question Answering over Knowledge Bases, ICLR2023
- Knowledge Solver: Teaching LLMs to Search for Domain Knowledge from Knowledge Graphs, Arxiv
- KnowledGPT: Enhancing Large Language Models with Retrieval and Storage Access on Knowledge Bases, Arxiv
- MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Models, Arxiv
- Augmenting Black-box LLMs with Medical Textbooks for Clinical Question Answering, Arxiv
- Chain of Knowledge: A Framework for Grounding Large Language Models with Structured Knowledge Bases, Arxiv
- KITLM: Domain-Specific Knowledge InTegration into Language Models for Question Answering, Arxiv
- RET-LLM: Towards a General Read-Write Memory for Large Language Models, Arxiv
- ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings, Arxiv, 2023
- Leveraging A Medical Knowledge Graph into Large Language Models for Diagnosis Prediction, Arxiv
- Knowledge-tuning Large Language Models with Structured Medical Knowledge Bases for Reliable Response Generation in Chinese, Arxiv
- Two is Better Than One: Answering Complex Questions by Multiple Knowledge Sources with Generalized Links, Arxiv
knowlege injection with external knowledge (document corpus or other types of knowledge that is different from KB)
The method in this part mainly integrates externally available domain knowledge into LLMs. Note that this external knowledge is different from the knowledge graph. It is generally presented in the form of natural text (not a graph), so there is no need to use graphs to retrieve or query.
- Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Framework, ACL2023
- Unified Demonstration Retriever for In-Context Learning, ACL2023
- LLaMAIndex, Github, 2022
- LangChain, Github, 2022
- ChatDB: Augmenting LLMs with Databases as Their Symbolic Memory, Arxiv, 2023
- The CALLA Dataset: Probing LLMs' Interactive Knowledge Acquisition from Chinese Medical Literature, Arxiv, 2023
This part organizes the methods of model self-driving to obtain domain knowledge. Specifically, this type of method designs prompts to allow the LLM to generate the required domain-related text by itself, and then uses the generated text to perform in-domain tasks.
- Generate rather than Retrieve: Large Language Models are Strong Context Generators, ICLR2023
- Least-to-Most Prompting Enables Complex Reasoning in Large Language Models, ICLR2023
- Large Language Models Are Reasoning Teachers, ACL2023
- Symbolic Chain-of-Thought Distillation: Small Models Can Also “Think” Step-by-Step, ACL2023
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, Arxiv