This is a CI/CD strategy for a simple Node/Express application integrating CodeBuild, CodeDeploy, and CodePipeline. It is advisable to understand the purpose and functionality of these individual AWS services and how they fit together to create a CI/CD pipeline.
- Diagram Overview
- Getting Started
- Codebase File Structure
- CloudFormation Stacks
- CodeBuild Notes
- CodeDeploy Notes
- Troubleshooting
- Helpful Resources
npm install
Make sure your project successfully lints and tests with no errors. Mocha must be installed globally.
npm run lint
npm test
Commit this repo to GitHub and note the repo name and branch you'd like to track for deployment.
Retrieve a GitHub token that allows AWS to source and watch your repo.
You must also retrieve an existing key name for your EC2 instances (or create a new key in the EC2 console).
Create a file named parameters.json in the cloudformation-templates folder that follows the given pattern:
[
{
"ParameterKey": "GitHubToken",
"ParameterValue": <Your GitHub Token>
},
{
"ParameterKey": "GitHubRepoOwner",
"ParameterValue": <Your GitHub Name>
},
{
"ParameterKey": "GitHubRepoName",
"ParameterValue": <Name of the GitHub Repo>
},
{
"ParameterKey": "GitHubBranch",
"ParameterValue": <Name of the Branch>
},
{
"ParameterKey": "S3ArtifactStore",
"ParameterValue": "codepipeline-us-east-1-<Provide a unique identifier here>"
},
{
"ParameterKey": "EC2KeyName",
"ParameterValue": <Name of existing Key for EC2 ssh capability>
}
]
This script will create the necessary resources for your CodeBuild projects (test & deployment), CodeDeploy application/group, and CodePipeline. It is highly likely you will need to edit these resources for your particular app's needs. You can deploy all the resources and trigger pipeline with the following command:
npm run cloudformation
After all the AWS resources are successfully created, an Elastic Load Balancer URL will be returned in your terminal. This URL will not work until the CodePipeline has completed a full execution and successfully deployed.
You can also deploy individual parts of the CloudFormation stack with the following commands:
npm run cloudformation-build
npm run cloudformation-deploy
npm run cloudformation-pipeline
The root folder contains buildspec.yml and appspec.yml files for CodeBuild and CodeDeploy respectively. Note that .eslintrc.json is included for linting during the build/test process in the cloud. It should not be included in the .gitignore file for this reason (unless linting is done via an additional npm package and script). The root folder also includes standard Node/Git project files: package.json and .gitignore.
The cloudformation-templates folder contains the necessary CloudFormation templates for CodeBuild, CodeDeploy, and CodePipeline as well as parameters.json (which is .gitignored since it contains a GitHub token).
The scripts folder contains script files related to the project.
The test folder contains spec files for unit testing. In this boilerplate, Mocha/SuperTest are used as a testing suite.
All code related to the Node App should be located in the /src folder.
The code-build.yml CloudFormation template creates the following resources:
- IAM Role for the CodeBuild project with attached policy
- CodeBuild project for testing with configurable lifecycle commands. You'll likely want to edit the commands (see CodeBuild Notes) and environment (Node version, Linux Container).
- CodeBuild project for deployment
The code-deploy.yml CloudFormation template creates the following resources:
- Instance Security Group with ports 22 and 80 availaible to all IPs (You'll likely want to change these security protocols.)
- IAM Role for EC2 instance with IAM Policy allowing Get and List access to S3
- EC2 Instance Profile with aforementioned IAM Role attached
- EC2 LaunchConfiguration with UserData script that installs Ruby, AWS-CLI, CodeDeploy software, and NodeJS (You'll likely want to edit the InstanceType and configure additional parameters for your specific project.)
- AutoScalingGroup with tags necessary for CodeDeploy (You'll likely want to edit MinSize, MaxSize, Availability Zones, and configure additional parameters for your specific project)
- Elastic Load Balancer connected to Autoscaling Group, listening on port 80
- IAM Role and Policy for CodeDeploy
- CodeDeploy Application with EC2 TagFilters referencing the tags attached to the AutoScaling Group
The code-pipeline.yml CloudFormation template creates the following resources:
- S3 Bucket to hold source code and build artifacts
- IAM Role for CodePipeline with attached IAM Policy
- CodePipeline with following stages
- Source (connects to GitHub Repo and Branch)
- Test (CodeBuild project named RepoName-BranchName-Test) which can be configured in the code-build.yml Cloudformation template under the resource property: TestCodeBuildProject
- Build (CodeBuild project named RepoName-BranchName) which is configured by the buildspec.yml in the root folder and the code-build.yml Cloudformation template under the resource property: CodeBuildProject
- Deploy (CodeDeploy application named RepoName-BranchName) which is configured by appspec.yml and the code-deploy.yml Cloudformation template resource properties: DeployApplication and DeploymentGroup
There two CodeBuild projects in the CodePipeline created by the Cloudformation templates: a test build and a build for deployment.
In order to configure the test build, most of the options are located in the code-build.yml template under the resource name: TestCodeBuildProject. In the Source.BuildSpec.phases properties, you are able to perform multiple commands in different parts of the build's lifecycle. This boilerplate includes:
- Install phase:
- npm install
- npm install -g mocha
- Pre_build phase:
- npm run lint
- Build phase
- npm test
Worth noting: if any of these lifecycle commands produces an error, the build will fail and the pipeline will stop.
If the test build passes, the pipeline continues with the deployment build. The CodeBuild project looks at the buildspec.yml located in the root folder for lifecycle commands. The build exports the necessary artifacts to the S3 bucket created in the codepipeline.yml template. In this boilerplate, all the folders/files are included recusively '**/*' as artifacts of the build. If you only want to export specific files or folders from the build phase for the deployment phase, they can be declared in the artifacts property of the buildspec.yml.
Deployment will largely depend on the needs of your specific app. In this boilerplate, an autoscaling group of EC2 instances is deployed. The configuration of your deployment is largely defined in the code-deploy.yml template. It's important to note that if you change the name of your deployment application or deployment group, that change will need to be reflected in the 'Deploy' phase of your CodePipeline, located in codepipeline.yml.
The AWS Console is useful for troubleshooting errors in the pipeline process. To debug any issues in the creation of your CloudFormation resources, use the CloudFormation console. If the CloudFormation templates have been succesfully created, the CodePipeline console helps debug errors that occur during the source/test/build/deploy phases of the pipeline. If you find that your deploy phase is failing, the best pace to troubleshoot is the CodeDeploy console. The instance 'events' logging feature is particularly useful. You can find it by navigating to the CodeDeploy Console > Deployments > Click on Deployment ID > Click 'View Events' for the latest Instance ID.
CodeDeploy Cloudformation Docs