/genai-llm-rag-pattern

Primary LanguagePythonApache License 2.0Apache-2.0


Logo

Validated Pattern for Retrieval-Augmented Generation (RAG) with LLM


Explore the docs »

View Demo · Report Bug · Request Feature

Table of Contents
  1. About The Project
  2. Getting Started
  3. Contributing
  4. License

About The Project

This project is a reference pattern for deploying an Enterprise Search Assistant based on Retrieval-Augmented Generation (RAG) with Large Language Models (LLMs), built on top of Red Hat OpenShift AI. By integrating a dynamic retrieval mechanism, it enables the chosen LLM to pull relevant information from extensive data sources stored in a vector databse, enhancing response accuracy and relevance. This approach not only broadens the model's knowledge base beyond its initial training but also ensures that its outputs are informed by the most current data available as the vector database contains a specialised index of enterprise documents which provide 'context' to the LLM allowing more accurate results but also allowing users to refer to source documents.

(back to top)

Getting Started

This project uses validated patterns to allow consistent deployment across multiple platforms. The project itself was initially forked off of the core mutlicloud-gitops pattern.

Prerequisites

This is an example of how to list things you need to use the software and how to install them.

  • An OpenShift Cluster with

    • Cluster admin rights logged in with oc where the repository is cloned out
    • Connectivity through to GitHub (or an equivalent git platform)
    • GPUs on at least one node within the cluster.

  • The current demonstration relies on flash-attention to decrease memory consumption for the LLM models. In order to run end-to-end demonstrations on this pattern, it is therefore recommended to have access to specific Nvidia GPUs which provide this capability, including:

    • Nvidia L40S
    • Nvidia A100
    • Nvidia H100/H200

Installation

The deployment method for validated pattenrs is described here.

(back to top)

Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!

  1. Fork the Project
  2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
  3. Commit your Changes (git commit -m 'Add some AmazingFeature')
  4. Push to the Branch (git push origin feature/AmazingFeature)
  5. Open a Pull Request

(back to top)

License

Distributed under the Apache-2.0 License. See LICENSE for more information.

(back to top)