s3-user-script

Securely bootstrap an EC2 instance using IAM Role credentials to download and run a User Data script from a private S3 bucket.

Getting started

Precompiled binaries

Linux (64-bit)

Compile from source

$ go get -u github.com/grosskur/s3-user-script

Background

The EC2 platform provides several features that can be used together to create elastic, dynamically configured clusters of machines:

User Data is arbitrary data that you can provide when you launch an instance. If this data is a shell script, it will be executed the first time the instance is booted.
A Launch Configuration is a template for launching repeated instances with the same parameters. It can also have User Data associated with it, which is passed on to each instance you launch.
An Auto-Scaling Group ties together a Launch Configuration to a Scaling Plan to let you dynamically grow or shrink a group of instances.
An IAM Role can be assigned to an instance or launch configuration to generate a temporary, automatically-rotated set of AWS credentials for that particular instance.

Problem

User scripts work fine when launching a single instance. However, when used with an auto-scaling group, you are essentially "baking" the data up-front into all the instances you will launch. The only way to change the user data is to destroy and recreate the launch configuration associated with the auto-scaling group.

Solution

s3-user-script is a shim that simply downloads the real user script from an S3 bucket and runs it. Since the S3 bucket should be private, IAM role credentials are used to access it. And to keep things simple, it assumes your user scripts are organized based on the role name (although this is configurable).

Usage

Create an S3 bucket my-user-scripts.

Create an IAM role MyRole. Give it access to your bucket with a policy like the following:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::my-user-scripts",
        "arn:aws:s3:::my-user-scripts/*",
      ]
    }
  ]
}

Create a user-script and upload it to s3://my-user-scripts/MyRole/user-script.

Create your instance with the following user data:

#!/bin/bash -e
curl -fLOsS https://github.com/grosskur/s3-user-script/releases/download/v20140226/s3-user-script
chmod 755 s3-user-script
exec ./s3-user-script -b my-user-scripts

Alternatively, if you bake /usr/local/bin/s3-user-script into your AMI (using a tool like Packer), your user data becomes even simpler:

#!/bin/bash -e
exec s3-user-script -b my-user-scripts

Congratulations! Your EC2 instances will now run the latest version of your role-specific user scripts on boot. Changes to the user scripts go live immediately when you update them on S3.