GenAI, from your Local machine to AWS with LangChain.js OR LlamaIndex.TS

⭐ If you like this sample, Star it on GitHub — Feedback is a Gift!

Overview • Get started • Resources • Troubleshooting

This sample shows how to build a Serverless GenAI chat application with Retrieval-Augmented Generation (RAG) and Agents using LangChain.js OR LlamaIndex.TS and AWS. The application is hosted on AWS Amplify and leverage Amazon Bedrock features like Knowledge Base and Agents, with Amazon OpenSearch Serverless as the vector database. Feel free to use it as a starting point to experiment advanced GenAI features within your own application.

Tip

You can test this application locally without any cost using Ollama. Follow the instructions in the Use your Local machine section to get started.

Overview

Building AI applications can be complex and time-consuming, but using LangChain.js OR LlamaIndex.TS and AWS Services allows to greatly simplify the process. This application is a chatbot that uses Formula 1 public datasets to generate responses to user queries.

We use a fictitious company called DeepF1, and the experience aims to allow race engineers to get insights about drivers performance metrics and Formula 1 Championship (2022, 2023, 2024).

This application is made from multiple components:

A web app made of a user authentication + chat web component built with Vite and hosted on AWS Amplify. The code is located in the webapp folder.
A backend using Amazon Bedrock features (Knowledge Base, Agent) to ingest documents with AWS Lambda using LangChain.js OR LlamaIndex.TS to query DeepF1 knowledge base and generate responses to the race engineers chat queries. The code is located in the infra folder.
A database to store the text extracted from the documents and the vectors generated by Amazon Bedrock Knowledge Base, using Amazon OpenSearch Serverless. The code is located in the infra folder.
An Object storage to store the source documents, using Amazon S3. The code is located in the infra folder.

Features

Serverless Architecture: Utilizes AWS Lambda and AWS Amplify for a fully Serverless deployment.
Retrieval-Augmented Generation (RAG) + Agentic: Combines the power of Amazon Bedrock and LangChain.js OR LlamaIndex.TS to provide relevant and accurate responses.
Scalable and Cost-Effective: Leverages AWS Serverless offerings to provide a scalable and cost-effective solution.
Local Development: Supports local development using Ollama for testing without any Cloud costs.

Get started

There are multiple ways to get started with this project. You can experiment using your Local machine or you can deploy the sample to AWS

Prerequisites

You need to install the following tools to use that sample:

Important

AWS region currently supported by this sample is us-east-1 (N. Virginia).

Use your Local machine

In addition to the Prerequisites, you need to install the following tools to work on your local machine:

Ollama

If you have a machine with enough resources, you can experiment locally without using any Cloud resources. To do that, you first have to install Ollama and then run the following commands to download models on your machine:

ollama pull llama3.2
ollama pull nomic-embed-text

Note

The llama3.2 model download few gigabytes of data, so it can take some time depending on your internet connection.

Then, you have to get the project code:

Select the Code button, then the Local tab, and copy the URL of this repository.
Open a terminal and run this command to clone this repo: git clone <repo-url>

Once cloned, go to the langchain-poc OR llamaindex-poc folder depending on the framework you want to experiment and install NPM dependencies:

npm install

When NPM dependencies are installed, go to the src subfolder and execute the following command lines:

node 01_local_invoke_model.js
node 02_local_rag_ollama.js
node 03_local_rag_bedrock.js

Note

While local models usually works well enough to answer the questions, sometimes they may not be able to follow perfectly the advanced formatting instructions and follow-up questions. This is expected, and a limitation of using smaller local models.

Important

If you want to run node 03_local_rag_bedrock.js, make sure to edit the config.js file and to provide a knowledgeBaseId (see Deploy the Backend section). If not already done, set up the AWS CLI using an AWS IAM User or Role with sufficient IAM Permissions to invoke the Amazon Bedrock Foundation Model of your choice.

If you want to run the Frontend on your Local machine, go to the webapp folder and install NPM dependencies:

npm install

If not already done, configure the AWS Amplify CLI on your local machine:

amplify configure

Then, initialize your own AWS Amplify application:

amplify init

Here is an example of initializing the AWS Amplify application (make sure the Distribution Directory Path is set to dist):

? Enter a name for the environment dev
? Choose your default editor: Visual Studio Code
✔ Choose the type of app that you're building · javascript
Please tell us about your project
? What javascript framework are you using react
? Source Directory Path: src
? Distribution Directory Path: dist
? Build Command: npm run-script build
? Start Command: npm run-script start
Using default provider awscloudformation
? Select the authentication method you want to use: AWS profile

Add Authentication to ensure only race engineers (i.e: faked users) can access cutting-edge features of your web app:

amplify add auth

Here is an example of adding authentication to the AWS Amplify application:

Using service: Cognito, provided by: awscloudformation
The current configured provider is Amazon Cognito.
Do you want to use the default authentication and security configuration? Default configuration
Warning: you will not be able to edit these selections.
How do you want users to be able to sign in? Username
Do you want to configure advanced settings? No, I am done.

Once authentication added, open the AWS IAM Roles Console, look for the amplify-webapp-dev-[UNIQUE ID]-authRole IAM role and attach the following policies (permissions):

arn:aws:iam::aws:policy/AmazonBedrockFullAccess (only used to ease usage of this sample, do NOT use such policy in PROD)

Lastly, push your local AWS Amplify application to ensure its configuration can be in sync with the Cloud one:

amplify push

When prompted if you want to continue, enter Y.

Once your configuration pushed to the Cloud, you can run the Frontend on your Local machine:

npm run dev

The Frontend will be available on http://localhost:3000

Deploy the sample to AWS

AWS prerequisites

AWS account. If you're new to AWS, create an AWS account.
AWS account permissions. Get sufficient AWS IAM permissions (in respect of the fine-grained permissions best practice) to deploy ALL AWS resources involved in this sample.
Amazon Bedrock with model access granted for the Foundation Model of your choice. You can request access by following instructions provided within the Amazon Bedrock User Guide.

Cost estimation

Pricing varies per region and usage, so it isn't possible to predict exact costs for your usage. However, you can use the AWS Pricing Calculator for the resources below to get an estimate.

⚠️ To avoid unnecessary costs, remember to delete ALL AWS resources if it's no longer in use (cf. Clean up section).

Deploy the Backend

Open a terminal and navigate to the infra folder.
Install NPM dependencies:

npm install

If not already done, set up the AWS CLI using an AWS IAM User or Role with sufficient AWS IAM permissions to deploy the sample.
Execute the following script into your terminal to deploy backend resources to AWS:

./cdk-deploy-to.sh [AWS_ACCOUNT_ID] [AWS_REGION] [AWS_PROFILE_NAME](optional)

Note

Make sure to replace [AWS_ACCOUNT_ID] [AWS_REGION] [AWS_PROFILE_NAME](optional) with your own AWS account Id and the AWS region targeted for your deployment.

This will provision all AWS backend resources (Amazon Bedrock, Amazon S3, AWS Lambda) and build the search index (Amazon OpenSearch Serverless) based on the files found in the ./data subfolder.

When prompted if you want to deploy these changes, enter y. The deployment process will take few minutes.

Deploy the Frontend

Once the Backend deployed with success, go to the the webapp folder within your terminal and execute the following actions:

Install NPM dependencies:

npm install

Open the src/App.tsx file and replace the bedrock section with the following one:

bedrock: {
    region: aws_exports.aws_project_region,
    // modelId: "meta.llama3-2-3b-instruct-v1:0",
    agent: {
        agentId: "[AMAZON BEDROCK AGENT ID]",
        agentAliasId: "[AMAZON BEDROCK AGENT ALIAS ID]"
    }
},

Important

Make sure to replace [AMAZON BEDROCK AGENT ID] and [AMAZON BEDROCK AGENT ALIAS ID] with Outputs values displayed in your terminal. Here is an example of Outputs:

DeepF1GenAIStack.AgentAliasIdOutput = 7J3ABCDEJG
DeepF1GenAIStack.AgentIdOutput = HGYUARPLMN

If not already done, configure the AWS Amplify CLI on your local machine to deploy this sample Frontend:

amplify configure

Initialize your own AWS Amplify application:

amplify init

Here is an example of initializing the AWS Amplify application (make sure the Distribution Directory Path is set to dist):

? Enter a name for the environment dev
? Choose your default editor: Visual Studio Code
✔ Choose the type of app that you're building · javascript
Please tell us about your project
? What javascript framework are you using react
? Source Directory Path: src
? Distribution Directory Path: dist
? Build Command: npm run-script build
? Start Command: npm run-script start
Using default provider awscloudformation
? Select the authentication method you want to use: AWS profile

Add Authentication to ensure only race engineers (i.e: faked users) can access cutting-edge features of your web app:

amplify add auth

Here is an example of adding authentication to the AWS Amplify application:

Once authentication added, open the AWS IAM Roles Console, look for the amplify-webapp-dev-[UNIQUE ID]-authRole IAM role and attach the following policies (permissions):

arn:aws:iam::aws:policy/AmazonBedrockFullAccess (only used to ease usage of this sample, do NOT use such policy in Production)

Add Hosting to publish your web app:

amplify add hosting

Open the src subfolder and rename the aws-exports.js file with the following name: aws-exports.ts
Publish your web app (on the Internet):

amplify publish

When prompted if you want to publish the frontend, enter Y. The deployment process will take few minutes.

Once built and artifacts published, you can use the public URL displayed in your terminal to open the DeepF1 Race AI Engineer web app and start chatting with the Foundation Model you specified (llama3.2 by default) which will leverage Agentic approach when required.

Clean up

To clean up ALL AWS resources deployed by the sample:

Go to the infra folder
Execute the following script into your terminal to destroy all stacks deployed by AWS CDK:

./cdk-destroy-from.sh [AWS_ACCOUNT_ID] [AWS_REGION] [AWS_PROFILE_NAME](optional)

When prompted if you want to delete all stacks, enter y.

Go the the webapp folder within your terminal:
Run the following command:

amplify delete

When prompted if you want to continue with the deletion of all the environments and local files created by Amplify CLI, enter y.

ALL AWS resources provisioned via the Deploy the sample to AWS section will be deleted.

Resources

Here are some resources to learn more about the technologies used in this sample:

You can also find more AWS samples here.

Troubleshooting

If you have any issue when running or deploying this sample, please open an issue in this repository.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of AWS Trademarks or logos is subject to and must follow AWS Trademark Guidelines & License Terms. Use of AWS trademarks or logos in modified versions of this project must not cause confusion or imply AWS sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

ajohn-wick/deepf1-ai-engineer

GenAI, from your Local machine to AWS with LangChain.js OR LlamaIndex.TS

Overview

Features

Get started

Prerequisites

Use your Local machine

Deploy the sample to AWS

AWS prerequisites

Cost estimation

Deploy the Backend

Deploy the Frontend

Clean up

Resources

Troubleshooting

Trademarks