GenAIInfra

GenAIInfra is the containerization and cloud native suite for OPEA, including artifacts to deploy GenAIExamples in a cloud native way, which can be used by enterprise users to deploy to their own cloud.

Overview

The GenAIInfra repository is organized under four main directories, which include artifacts for OPEA deploying:

Directory	Purpose
`helm-charts`	Helm charts for deploying GenAIExamples on Kubernetes.
`kubernetes-addons`	Deploy Kubernetes add-ons for OPEA.
`manifests`	Manifests for deploying GenAIExamples on Kubernetes and on Docker Compose.
`scripts`	Scripts for testing, tools for facilitate OPEA deployment, and etc.

Prerequisite

GenAIInfra uses Kubernetes as the cloud native infrastructure. Please follow the steps below to prepare the Kubernetes environment.

Setup Kubernetes cluster

Please follow Kubernetes official setup guide to setup Kubernetes. We recommend to use Kubernetes with version >= 1.27.

There are different methods to setup Kubernetes production cluster, such as kubeadm, kubespray, and more.

NOTE: We recommend to use containerd when choosing the container runtime during Kubernetes setup. Docker engine is also verified on Ubuntu 22.04 and above.

(Optional) To run GenAIInfra on Intel Gaudi product:

The following steps are optional. They're only required if you want to run the workloads on Intel Gaudi product.

Please check the support matrix to make sure that environment meets the requirements.
Install Intel Gaudi software stack.
Install and setup container runtime, based on the container runtime used by Kubernetes.

NOTE: Please make sure you configure the appropriate container runtime based on the type of container runtime you installed during Kubernetes setup.

Install Intel Gaudi device plugin for Kubernetes.

Usages

Use helm charts to deploy

To deploy GenAIExamples to Kubernetes using helm charts, you need Helm installed on your machine.

Clone the GenAIInfra repository and change into the helm-charts directory:

git clone https://github.com/opea-project/GenAIInfra.git
cd GenAIInfra/helm-charts

Select the example workflow you want to deploy, set the customized values in values.yaml and deploy the example (e.g. codegen) using helm:

helm install codegen ./codegen

Use manifests to deploy

GenAIInfra also supports deploy GenAIExamples using manifests, you need kubectl installed on your machine.

Clone the GenAIInfra repository and change into the manifests directory:

git clone https://github.com/opea-project/GenAIInfra.git
cd GenAIInfra/manifests

Select the example workflow you want to deploy, deploy the example (e.g. DocSum) using kubectl:

kubectl apply -f ./DocSum/manifests/

gyohuangxin/GenAIInfra