NOTE: Same reference architecture, but using AWS Fargate is available in fargate branch

Blue/Green deployments on ECS

This reference architecture is in reference to blog post on blue green deployments on ECS. It creates a continuous delivery by leveraging AWS CloudFormation templates. The templates creates resources using Amazon's Code* services to build and deploy containers onto an ECS cluster as long running services. It also includes a manual approval step facilitated by lambda function that discovers and swaps target group rules between 2 target groups, promoting the green version to production and demoting the blue version to staging.

Pre-Requisites

This example uses AWS Command Line Interface to run Step-3 below.

Please follow instructions if you haven't installed AWS CLI. Your CLI configuration need PowerUserAccess and IAMFullAccess IAM policies associated with your credentials

aws --version

Output from above must yield AWS CLI version >= 1.11.37

Quick setup in three steps

1. Fork ECS Sample app

Fork the Amazon ECS sample app GitHub repository into your GitHub account.

Clone the ECS Sample app repo

git clone https://github.com/<your_github_username>/ecs-demo-php-simple-app

2. Clone ECS blue green repo

git clone https://github.com/awslabs/ecs-blue-green-deployment

2a (Optional) . Switch to fargate branch , if you want to use AWS Fargate

git checkout fargate

3. Run bin/deploy

bin/deploy

Here are the inputs required to launch CloudFormation templates:

S3 Bucket: Enter S3 Bucket for storing your CloudFormation templates and scripts. This bucket must be in the same region where you wish to launch all the AWS resources created by this example.
CloudFormation Stack Name: Enter CloudFormation Stack Name to create stacks
GitHubUser: Enter your GitHub Username
GitHubToken: Enter your GitHub Token for authentication (https://github.com/settings/tokens)

Sit back and relax until all the resources are created for you. After the templates are created, you can open ELB DNS URL to see the ECS Sample App

For testing Blue Green deployment, Go ahead and make a change in ECS Sample App. For ex, edit src/index.php and update the background-color to #20E941 to change to Green background color. After commiting to your repo, Code Pipeline will pick the change automatically and go through the process of updating your application.

Click on "Review" button in Code pipeline management console and Approve the change. Now you should see the new version of the application with Green background.

Resources created in this exercise

Count	AWS resources
7	AWS CloudFormation templates
1	Amazon VPC (10.215.0.0/16)
1	AWS CodePipeline
2	AWS CodeBuild projects
1	Amazon S3 Bucket
1	AWS Lambda
1	Amazon ECS Cluster
2	Amazon ECS Service
1	Application Load Balancer
2	Application Load Balancer Target Groups

Implementation details

During first phase, the parent template (ecs-blue-green-deployment.yaml) kicks off creating VPC and the resources in deployment-pipeline template. This creates CodePipeline, CodeBuild and Lambda resources. Once this is complete, second phase creates the rest of resources such as ALB, Target Groups and ECS resources. Below is a screenshot of CodePipeline once all CloudFormation templates are completed

The templates create two services on ECS cluster and associates a Target Group to each service as depicted in the diagram. Blue Target Group is associated with Port 80 that represents Live/Production traffic and Green Target Group is associated with Port 8080 and is available for new version of the Application. The new version of the application can be tested by accessing the load balancer at port 8080, example http://LOAD_BALANCER_URL:8080 .If you want to restrict the traffic ranges accessing beta version of the code, you may modify the Ingress rules here. During initial rollout, both Blue and Green service serve same application versions.As you introduce new release, CodePipeline picks those changes and are pushed down the pipeline using CodeBuild and deployed to the Green service. In order to switch from Green to Blue service (or from beta to Prod environment), you have to Approve** the release by going to CodePipeline management console and clicking Review** button. Approving the change will trigger Lambda function (blue_green_flip.py) which does the swap of ALB Target Groups. If you discover bugs while in Production, you can revert to previous application version by clicking and approving the change again. This in turn will put Blue service back into Production. To simplify identifying which Target Groups are serving Live traffic, we have added Tags on ALB Target Groups. Target Group IsProduction Tag will say true for Production application.

Here is further explaination for each stages of Code Pipeline.

During Build stage

During first phase, CodeBuild builds the docker container image and pushes to Amazon ECR.
During second phase, Codebuild executes scripts/deployer.py which executes the following scripted logic
1. Retrieve artifact (build.json) from the previous phase (CodeBuild phase, which builds application container images)
2. Check if the load balancer exists. Name of the ELB is fed through environment variable by the pipeline.
3. Get tag key value of the target group, running on port 8080 and 80 with KeyName as "Identifier". It will be either "Code1" or "Code2"
4. Get Sha of the image id running on target group at port 8080 and 80
5. Edit the build.json retrieved from step-1 and append the values retrieved in step3 and step4
6. Save the modified build.json. This file is the output from codebuild project and fed as an input to the CloudFormation execution stage.This json file has the following schema { "Code1" : "CONTAINER_TAG1", "Code2" : "CONTAINER_TAG2" } If the load balancer does not exists (as found in step-2), this would imply that the stack is executed for the first time, and the values of "CONTAINER_TAG1" and CONTAINER_TAG2" will be the same and default to the value retrieved from build.json in step-1

During Deploy stage CodePipeline executes templates/ecs-cluster.yaml. The CloudFormation input parameters with KeyName as "Code1" and "Code2" are overwritten with the values as written in the build.json, retrieved from the second phase of Build Stage.

During Review stage The pipeline offers manual "Review" button so that the approver can review code and Approve new release. Providing approvals at this stage will trigger the Lambda function (blue_green_flip.py) which swaps the Green Target Group to Live traffic. You can checkout sample app to see new release change. blue_green_flip.py has the following logic scripted

Read Job Data from input json
Read Job ID from input json
Get parameters from input json
Get Load balancer name from parameters
Identify the TargetGroup running on this Load Balancer at port 80 and port 8080. Perform the TargetGroup Swap. Also swap the values of "IsProduction" tags.
Send success or failure to CodePipeline

Cleanup

First delete ecs-cluster CloudFormation stack, this will delete both ECS services (BlueService and GreenService) and LoadBalancer stacks. Next delete the parent stack. This should delete all the resources that were created for this exercise

dev-usa/ecs-blue-green-deployment