/MLOps-W8

Primary LanguagePureBasic

Week 8 - Deploying Pet-Bokeh using Kubernetes with Nvidia Triton on EC2

📚 Learning Objectives

By the end of this session, you will be able to:

  • Create and configure a minikube cluster
  • Integrate minikube with Nvidia Trition Server
  • Deploy a minikube cluster on EC2

📦 Deliverables

  • A screenshot of kubectl dashboard
  • A screenshot of your deployment

Deployment on EC2

Create EC2 Instance

Install dependencies

  • Get the ip address of the instance
  • Change key permissions to 400 (chmod 400 key.pem)
  • SSH into the machine ssh -i key.pem ec2-user@ec2.ip.address
  • Install git if needed (sudo apt install git for ubuntu based distros, sudo yum install git for amazon linux)
  • Install Docker (sudo apt install docker for ubuntu based distros, sudo yum install docker for amazon linux)
  • Start Docker (sudo systemctl start docker)
  • Add user to docker group (sudo usermod -aG docker ${USER})
  • Logout and Login again through SSH to take the group changes into account
  • Check if docker installed correctly (docker run hello-world)
  • Install minikube
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
sudo install minikube-linux-amd64 /usr/local/bin/minikube
  • Start a minikube cluster
  • Set an alias for kubectl
  • For triton we need to add the credentials as secrets
kubectl create secret generic aws-env --from-literal='AWS_ACCESS_KEY_ID=YOUR_ACCESS_KEY' --from-literal='AWS_SECRET_ACCESS_KEY=YOUR_SECRET_ACCESS_KEY' --from-literal='AWS_DEFAULT_REGION=us-east-1'

Locally built images

  • Use minikube's docker
  • Pull the triton image
  • Build all the docker images
docker build -t main .
docker build -t face-emotion face-emotion/
docker build -t pet-bokeh pet-bokeh/
  • Load all the kubernetes resources
  • Forward the main port (kubectl port-forward svc/main 8004:8004 --address 0.0.0.0)

Pulling from ECR

  • Configure credentials
  • Enable the addon
  • Edit K8s/ECR/*.yaml to use your ECRs
  • Load all the kubernetes resources
  • Forward the main port (kubectl port-forward svc/main 8004:8004 --address 0.0.0.0)