I am pleased to present this comprehensive collection of advanced Retrieval-Augmented Generation (RAG) techniques. The aim is to provide a valuable resource for researchers and practitioners seeking to enhance the accuracy, efficiency, and contextual richness of their RAG systems.
Retrieval-Augmented Generation (RAG) is revolutionizing the way we combine information retrieval with generative AI. This repository showcases a curated collection of advanced techniques designed to supercharge your RAG systems, enabling them to deliver more accurate, contextually relevant, and comprehensive responses.
- 🧠 State-of-the-art RAG enhancements
- 📚 Comprehensive documentation for each technique
- 🛠️ Practical implementation guidelines
- 🌟 Regular updates with the latest advancements
Explore the extensive list of cutting-edge RAG techniques:
1. Simple RAG 🌱
Introducing basic RAG techniques ideal for newcomers.
Start with basic retrieval queries and integrate incremental learning mechanisms.
Enhancing retrieval accuracy by embedding individual sentences and extending context to neighboring sentences.
Retrieve the most relevant sentence while also accessing the sentences before and after it in the original text.
Applying various filtering techniques to refine and improve the quality of retrieved results.
- 🏷️ Metadata Filtering: Apply filters based on attributes like date, source, author, or document type.
- 📊 Similarity Thresholds: Set thresholds for relevance scores to keep only the most pertinent results.
- 📄 Content Filtering: Remove results that don't match specific content criteria or essential keywords.
- 🌈 Diversity Filtering: Ensure result diversity by filtering out near-duplicate entries.
Optimizing search results by combining different retrieval methods.
Combine keyword-based search with vector-based search for more comprehensive and accurate retrieval.
Applying advanced scoring mechanisms to improve the relevance ranking of retrieved results.
- 🧠 LLM-based Scoring: Use a language model to score the relevance of each retrieved chunk.
- 🔀 Cross-Encoder Models: Re-encode both the query and retrieved documents jointly for similarity scoring.
- 🏆 Metadata-enhanced Ranking: Incorporate metadata into the scoring process for more nuanced ranking.
Modifying and expanding queries to improve retrieval effectiveness.
- ✍️ Query Rewriting: Reformulate queries to improve retrieval.
- 🔙 Step-back Prompting: Generate broader queries for better context retrieval.
- 🧩 Sub-query Decomposition: Break complex queries into simpler sub-queries.
Creating a multi-tiered system for efficient information navigation and retrieval.
Implement a two-tiered system for document summaries and detailed chunks, both containing metadata pointing to the same location in the data.
Generating hypothetical questions to improve alignment between queries and data.
Create hypothetical questions that point to relevant locations in the data, enhancing query-data matching.
Selecting an appropriate fixed size for text chunks to balance context preservation and retrieval efficiency.
Experiment with different chunk sizes to find the optimal balance between preserving context and maintaining retrieval speed for your specific use case.
Dividing documents based on semantic coherence rather than fixed sizes.
Use NLP techniques to identify topic boundaries or coherent sections within documents for more meaningful retrieval units.
Compressing retrieved information while preserving query-relevant content.
Use an LLM to compress or summarize retrieved chunks, preserving key information relevant to the query.
Providing transparency in the retrieval process to enhance user trust and system refinement.
Explain why certain pieces of information were retrieved and how they relate to the query.
Implementing mechanisms to learn from user interactions and improve future retrievals.
Collect and utilize user feedback on the relevance and quality of retrieved documents and generated responses to fine-tune retrieval and ranking models.
Dynamically adjusting retrieval strategies based on query types and user contexts.
Classify queries into different categories and use tailored retrieval strategies for each, considering user context and preferences.
Performing multiple rounds of retrieval to refine and enhance result quality.
Use the LLM to analyze initial results and generate follow-up queries to fill in gaps or clarify information.
Combining multiple retrieval models or techniques for more robust and accurate results.
Apply different embedding models or retrieval algorithms and use voting or weighting mechanisms to determine the final set of retrieved documents.
Incorporating structured data from knowledge graphs to enrich context and improve retrieval.
Retrieve entities and their relationships from a knowledge graph relevant to the query, combining this structured data with unstructured text for more informative responses.
Extending RAG capabilities to handle diverse data types for richer responses.
Integrate models that can retrieve and understand different data modalities, combining insights from text, images, and videos.
Implementing a recursive approach to process and organize retrieved information in a tree structure.
Use abstractive summarization to recursively process and summarize retrieved documents, organizing the information in a tree structure for hierarchical context.
20. Self RAG 🔁
A dynamic approach that combines retrieval-based and generation-based methods, adaptively deciding whether to use retrieved information and how to best utilize it in generating responses.
• Implement a multi-step process including retrieval decision, document retrieval, relevance evaluation, response generation, support assessment, and utility evaluation to produce accurate, relevant, and useful outputs.
21. Corrective RAG 🔧
A sophisticated RAG approach that dynamically evaluates and corrects the retrieval process, combining vector databases, web search, and language models for highly accurate and context-aware responses.
• Integrate Retrieval Evaluator, Knowledge Refinement, Web Search Query Rewriter, and Response Generator components to create a system that adapts its information sourcing strategy based on relevance scores and combines multiple sources when necessary.
An advanced RAG solution designed to tackle complex questions that simple semantic similarity-based retrieval cannot solve. This approach uses a sophisticated deterministic graph as the "brain" 🧠 of a highly controllable autonomous agent, capable of answering non-trivial questions from your own data.
• Implement a multi-step process involving question anonymization, high-level planning, task breakdown, adaptive information retrieval and question answering, continuous re-planning, and rigorous answer verification to ensure grounded and accurate responses.
To start implementing these advanced RAG techniques in your projects:
- Clone this repository:
git clone https://github.com/NirDiamant/RAG_Techniques.git - Navigate to the technique you're interested in:
cd rag-techniques/technique-name - Follow the detailed implementation guide in each technique's directory
We welcome contributions from the community! If you have a new technique or improvement to suggest:
- Fork the repository
- Create your feature branch:
git checkout -b feature/AmazingFeature - Commit your changes:
git commit -m 'Add some AmazingFeature' - Push to the branch:
git push origin feature/AmazingFeature - Open a pull request
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
⭐️ If you find this repository helpful, please consider giving it a star!
Keywords: RAG, Retrieval-Augmented Generation, NLP, AI, Machine Learning, Information Retrieval, Natural Language Processing, LLM, Embeddings, Semantic Search