nadaahm/awesome-ec2-spot

A curated list of awesome AWS EC2 Spot related updates, open source repos, guides, blogs, and other resources.

Awesome Amazon EC2 Spot Instances

A curated list of awesome Amazon EC2 Spot related tools, open source repos, guides, blogs, and other resources.

Contributing

Contributions are welcome!

Let's get srtarted!

Contents

• Hands-on
• Workloads
  • General
  • Data Analytics
  • Containers
    • Amazon EKS and Kubernetes
    • Amazon ECS and Fargate Spot
  • Web Applications
  • CI/CD
  • Machine Learning
  • Autonomous Vehicle
• Tools
• Tutorials
• Customer Stories
• News and Announcements

Hands-on

• https://ec2spotworkshops.com – 90mins use-case focused self-paced workshops.
• https://github.com/awslabs/ec2-spot-labs – Sample code snippets to get started with Spot.

Workloads

- General

• Best practices for handling EC2 Spot Instance interruptions
• Introducing Instance Refresh for EC2 Auto Scaling

- Data Analytics

• [Article] Best practices for running Apache Spark applications using Amazon EC2 Spot Instances with Amazon EMR
• [Video] Best practices for running Spark applications using Spot instances on Amazon EMR

- Containers

Amazon EKS and Kubernetes

• [Article] Building for Cost optimization and Resilience for EKS with Spot Instances
• [Video] Save costs running Kubernetes clusters with EC2 Spot instances
• [Video] Optimize AI/ML, Kubernetes and Big data workloads at your startup

Amazon ECS and Fargate Spot

• Leverage EC2 Spot with ECS Capacity Providers to manage compute capacity and scaling for your cluster.
• Handle interruptions, use automated draining for Spot instance in ECS cluster.
• [Article] Deep dive into Fargate Spot to run your ECS Tasks for up to 70% less.
• Monitor Fargate Spot task interruption notifications to ensure workload resiliency
• [Video] Using Fargate Spot to Optimize costs for serverless containers

- Web Applications

• [Article] Running Web Applications on Amazon EC2 Spot Instances
• [Video] How to build scalabe web applications with EC2 Spot

- CI/CD

• [Article] Cost Optimize your Jenkins CI/CD pipelines using EC2 Spot Instances
• [Video] Running Enteprise CI/CD workloads with Spot instances and Cloudbees

- Machine Learning

• [Article] Train Deep Learning Models on GPUs using Amazon EC2 Spot Instances
• [Article] TensorFlow Serving on Kubernetes with Amazon EC2 Spot Instances
• [Video] Reduce Inferencing costs by up to 90% Using Elastic Inference and Spot Instances

- Autonomous Vehicle

• [Article] Autonomous Vehicle and ADAS development on AWS Part 1: Achieving Scale

Tools

• Spot Instance Advisor
• AWS Node Termination Handler
• Amazon EC2 Instance Selector
• Amazon EC2 Metadata Mock
• AutoSpotting
• EC2 Spot Interruption Dashboard

Tutorials

• Amazon EKS with EC2 Spot Instances
• Running Batch jobs at scale for less with EC2 Spot instances
• Optimizing Amazon EMR clusters with EC2 Spot instances
• Optimizing and Scaling Machine Learning Training with managed Spot Trainig for Amazon Sagemaker

Customer Stories

• Examples of how customers have achieved business agility, cost savings, and scale with EC2 Spot instances.
• Capacity-Optimized Spot Instance allocation Strategy in action at Mobileye and Skyscanner
• Western Digital HDD Simulation at Cloud Scale – 2.5 Million HPC tasks, 40K EC2 Spot Instances
• Nearmap Builds AWS-Powered ML Pipeline to Help Others Analyze the World

News and Announcements

• [Jul-2020] Amazon EMR now supports Managed Scaling – automatically resizing clusters to lower cost
• [Jul-2020] AWS Marketplace AMIs now supported with Spot Instances in the EC2 Launch Instance Wizard
• [Jun-2020] Amazon EMR uses real-time capacity insights to provision spot instances to lower cost and interruption
• [Jan-2020] Amazon EC2 Spot instances can now be stopped and started similar to On-Demand instances
• [Dec-2019] Amazon ECS Capacity Providers Now Available