/serverless-rag-demo

Amazon Bedrock Anthropic Claude, (Llama2-7B/13B/70B /Falcon-7B/40B/180B on Amazon Sagemaker) LLMs with Amazon Opensearch Serverless as a Vector DB

Primary LanguagePythonMIT No AttributionMIT-0

Scalable RAG solutions/Agentic Workflows with Amazon Bedrock and Amazon Opensearch serverless service

Overview

Widespread AI adoption is being driven by generative AI models that can generate human-like content. However, these foundation models are trained on general data making it less effective for domain specific tasks. There lies the importance of Retrieval Augmented Generation (RAG). RAG allows augmenting prompts with relevant external data for better domain-specific outputs. With RAG, documents and queries are converted to embeddings, compared to find relevant context, and that context is appended to the original prompt before being passed to the LLM. Knowledge libraries can be updated asynchronously to provide the most relevant external data for augmenting prompts.

Amazon Opensearch Serverless(AOSS) offers vector engine to store embeddings for faster similarity searches. The vector engine provides a simple, scalable, and high-performing similarity search capability in Amazon OpenSearch Serverless that makes it easy for you to build generative artificial intelligence (AI) applications without having to manage the underlying vector database infrastructure.

Note

This repository offers a production ready easily deployable Generative AI solution with the below features:

  1. Document chat
  2. Multi-Agent collaboration
  3. Sentiment Analysis
  4. PII Redaction
  5. OCR

Important

The Older UI is maintained in the v0.0.1(Old-UI) branch.

Demos

Doc Chat/Doc Management (Multi-lingual)

output

Multi-Agent Demo

ma

PII Redaction
OCR

ocr

Sentiment Analysis
### Prerequisites
Prerequisites

Familiarity with below Services

Architecture

architecture

Deploying the Solution to your AWS account with AWS Cloudshell

Create an Admin User to deploy this stack

Section1 - Create an IAM user with Administrator permissions (OPTIONAL: If you're already an Admin role, you may skip this step)

  1. Search for the service IAM on the AWS Console and go the IAM Dashboard and click on “Roles“ tab under ”Access Management” and Click on “Create Role”
Screenshot 2024-04-05 at 5 52 42 PM
  1. Select AWS Account and click “Next“
role-iam
  1. Under permissions select Administrator access
Screenshot 2024-04-05 at 5 31 30 PM

  1. Give the role a name and create the role Screenshot 2024-04-05 at 5 33 45 PM

  2. You can now assume this role and proceed to deploy the stack. Click on Switch-Role

assune-role
  1. Switch role
Screenshot 2024-04-05 at 5 42 06 PM
  1. Proceed to cloudshell step
Deploy the RAG based Solution (Total deployment time 40 minutes)

Section 2 - Deploy this RAG based Solution (The below commands should be executed in the region of deployment)

  1. Switch to Admin role. Search for Cloudshell service on the AWS Console and follow the steps below to clone the github repository Screenshot 2024-04-05 at 5 48 41 PM

  2. Git Clone the serverless-rag-demo repository from aws-samples

     git clone https://github.com/aws-samples/serverless-rag-demo.git

  3. Go to the directory where we have the downloaded files.

      cd serverless-rag-demo

  4. Fire the bash script that creates the RAG based solution. Pass the environment and region for deployment. environment can be dev,qa,sandbox. Look at Prerequisites to deploy to the correct region.

      sh creator.sh

  5. Press Enter to proceed with deployment of the stack or ctrl+c to exit

    Cloudshell

  6. The UI is hosted on AppRunner the link to AppRunner could be found in CloudShell once the script execution is complete, or you could also go to the AppRunner service on the AWS Console and obtain the https url. The UI is authenticated through Amazon Cognito hence the very first time you would have to sign-up and then sign-in to login to the application AppRunnerUI

  7. On Amazon Bedrock page enable access to the below models model_access

(ADVANCED) Using an existing Bedrock Knowledge base

[!IMPORTANT] You could query your existing Knowledge base created on Amazon Bedrock provided it uses Amazon Opensearch Serverless service.

Steps

  1. Get the Collection ARN and the embedding model used by your Knowledge base on Bedrock Collection-ARN

  2. Head to Amazon Opensearch Serverless and search by ARN to fetch Opensearch Endpoint OSS-Endpoint

  3. Modify the configurations of your bedrock_rag_query_* lambda function. Set the below a. IS_BEDROCK_KB = yes
    b. OPENSEARCH_VECTOR_ENDPOINT = <> c. EMBED_MODEL_ID = <>. Find the base model Id from here (https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html) d. VECTOR_INDEX_NAME = <<VECTOR_INDEX used by your Bedrock KB>> e. BEDROCK_KB_EMBEDDING_KEY = <> Lambda-Config Bedrock-page

  4. Get the ARN of the Lambda role Lambda-Config

  5. Head to Amazon Opensearch on the AWS Console and click on Data Access Policies. Search for the Data Access Policy attached to your Bedrock KB and click on the Edit button Access-AOSS

  6. In the principal section add the ARN of your Lambda role and hit save Lambda-Access-AOSS

  7. Now try Document Chat on the UI, it should query from your Amazon Bedrock Knowledge base.

    [!IMPORTANT] We do not support indexing to an existing Knowledge base. That can be done through the Amazon Bedrock Console.