aws-solutions/qnabot-on-aws

Llama in Bedrock Knowledge base model ID

Closed this issue · 3 comments

preethy-1 commented

Hi,

In the CloudFormation stack in us-east-1, we could see only below list of models in the allowed list. Client requirement is to create knowledge base with Llama models. Can we update this allowed list and proceed? Will this works?
"anthropic.claude-instant-v1",
"anthropic.claude-v2.1",
"anthropic.claude-3-sonnet-v1",
"anthropic.claude-3-haiku-v1".

anjugds commented

We tried running the CF in us-east-1 with llama. CloudFormation ran successfully, but Bot responded with error. Could you please help.

fhoueto-amz commented

Hi @anjugds
The models listed in the dropdown are the models supported out of the box with the solution. If you want to bring your own model (llama or other), refer to https://github.com/aws-solutions/qnabot-on-aws/tree/main/source/docs/LLM_Retrieval_and_generative_question_answering and https://aws.amazon.com/blogs/machine-learning/delight-your-customers-with-great-conversational-experiences-via-qnabot-a-generative-ai-chatbot/, you can specify your llm or embeddings via a custom Lambda function.
Which version of the solution are you using? It is recommended to use the bug or feature template to create your issue so that you can provide relevant context information. The latest version of the solution 6.1.0 does provide support for meta-llama3-8b-instruct-v1. I would recommend that you either upgrade or write a custom lambda depending on your needs. i will close this. If you need any additional information please create an issue by using the new issue button and provide the appropriate context information.

preethy-1 commented

Hi @fhoueto-amz
Thanks for your response. We are specifically asking about this.

"BedrockKnowledgeBaseModel": {
"Type": "String",
"Description": "Required if BedrockKnowledgeBaseId is not empty. Sets the preferred LLM model to use with the Bedrock knowledge base. Please ensure you have requested access to the LLMs in Bedrock console (https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html), before deploying",
"AllowedValues": [
"amazon.titan-text-premier-v1",
"anthropic.claude-instant-v1",
"anthropic.claude-v2.1",
"anthropic.claude-3-sonnet-v1",
"anthropic.claude-3-haiku-v1"
],
"Default": "anthropic.claude-instant-v1"
}

Is it possible to include "meta.llama3-1-8b-instruct-v1:0" in BedrockKnowledgeBaseModel?