Load Testing SageMaker Endpoints

Setup

To get started have an EC2 instance up and running, in the case of this example we used an ml.c5.18xlarge instance. You can launch multiple EC2 instances to further get client side power to ramp up traffic. This script is not just EC2 reliant, if you want you can run this on EKS or even a SageMaker Notebook Instance, just be aware of the client compute power you have so that you do not get throttled on that end. For launching and setting up an EC2 instance follow this documentation.

locust folder contains distributed.sh which kicks of the distributed locust load test.

Load Test

Steps to run a load test:

  1. Update distributed.sh ENV variables:
export REGION=us-east-1
export CONTENT_TYPE=application/json
export PAYLOAD='{"inputs": "I am super happy right now."}'
export USERS=10
export WORKERS=30
export RUN_TIME=1m
  1. chmod +x distributed.sh
  2. ./distributed.sh endpoint-name

Security

See CONTRIBUTING for more information.

License

This library is licensed under the MIT-0 License. See the LICENSE file.