aws-samples/host-yolov8-on-sagemaker-endpoint

Host a YOLOv8 model on a SageMaker Endpoint

This aim of this project is to host a YOLOv8* PyTorch model on a SageMaker Endpoint and test it by invoking the endpoint. The project utilizes AWS CloudFormation/CDK to build the stack and once that is created, it uses the SageMaker notebooks created in order to create the endpoint and test it.

(*) NOTE: YOLOv8 is distributed under the GPLv3 license.

For YOLOv5 TensorFlow deployment on SageMaker Endpoint, kindly refer to the GitHub and the Blog on YOLOv5 on SageMaker Endpoint

AWS Architecture:

AWS CloudFormation Stack Creation

The AWS CloudFormation Stack can be created using 2 methods: (1) Using Template or (2) Using AWS CDK. Both the methods are described as follows:

1. Create Stack using AWS CloudFormation:

   • Choose Launch Stack and (if prompted) log into your AWS account:
   • Select a unique Stack Name, ackowledge creation of IAM resources, create the stack and wait for a few minutes for it to be successfully deployed
     1. 
     2. 
     3. 

2. PyTorch YOLOv8 model with AWS CDK In order to create the stack with AWS CDK, follow the steps highlighted in yolov8-pytorch-cdk. Use these steps:

$ cd yolov8-pytorch-cdk
$ python3 -m venv .venv
$ source .venv/bin/activate
$ pip3 install -r requirements.txt
$ cdk synth
$ cdk bootstrap
$ cdk deploy

YOLOv8 PyTorch model deployment on Amazon SageMaker Endpoints:

• From AWS Console, go to Amazon SageMaker Notebook Instances
• Select the Notebook created by the stack and open it
• Inside SageMaker Notebook, navigate: sm-notebook and open the notebooks: 1_DeployEndpoint.ipynb & 2_TestEndpoint.ipynb
  1. 1_DeployEndpoint.ipynb: Download YOLOv8 model, zip inference code and model to S3, create SageMaker endpoint and deploy it
  2. 2_TestEndpoint.ipynb: Test the deployed endpoint by running an image and plotting output; Cleanup the endpoint and hosted model

Contributors:

• Kevin Song (@kcsong)
• Romil Shah (@rpshah)