CDK Starter application to deploy Lambda via CodePipeline (with local CodeBuild and Typescript icing)
In this article, we create a starter AWS CDK project which deploys a CodePipeline for creating a NodeJS Lambda. This article is based on another article from AWS, but tweaks a number of things and builds on it, specifically including local CodeBuild functionality, as well as changing the Lambda language to TypeScript.
After starting a new endeavor mid last year working with AWS for the first time, one of the first projects that I took on was to create CloudWatch alarms and dashboards for the various AWS resources that our system was going to use. We were just going live for the first time, so it was critical to have the appropriate alarms in place to notify us if something was going wrong.
The system's existing resources were defined in CloudFormation templates, however, after some research, CDK (Cloud Development Kit) from AWS was the clear way to go forward. The AWS SDK and CDK make a good combination, especially if there's information you need on existing resources in order to create new resources -- you can use the SDK to first interrogate existing resources and then use the procured information in the CDK to build out more -- perfect for my use case.
I want to use the CDK for some future personal projects, so the idea of this post is to get a basic CDK app working that automatically updates resources when commits are made to a source repository.
The setup docs for CDK are easy to follow to install the CLI tools, setup your AWS credentials, and run "cdk init" to create a skeleton project in the language of your choice. Today we're using TypeScript.
A large portion of the code for my starter project is procured from this AWS CDK "CodePipeline Example" article (mentioned in intro). The AWS article includes basically just code snippets, instead of a fully functioning, downloadable repository. I've done that second part, which is the content of this repository on GitHub. The code in this aws-cdk-lambda-starter repo must be deployed to AWS CodeCommit (or you can integrate GitHub instead).
I liked starting with this example because, once the CDK app is deployed, the Lambda code will be deployed automatically via CodePipeline after any commits are made to the source repository. Basically, we have a CDK app and a NodeJS app (for the Lambda, in a subfolder) in one repository. There are two CloudFormation stacks that are defined.
The CDK app "bootstraps" with the first PipelineDeployingLambdaStack. We have to run "cdk deploy" on local machine to deploy this into the AWS account.
After that PipelineDeployingLambdaStack deploys, it creates a CodePipeline, which is triggered anytime new commits are made to the source repository.
The CodePipeline contains two CodeBuild projects: the first CodeBuild project builds the code that is in the "polly-client-lambda" directory, and the second creates a CF template via CDK, called "StackForLambdaDeployment" (I renamed it in my code"), which ultimately creates the Lambda from the previous CodeBuild. Once the CodePipeline finishes, the output of the two CodeBuild projects are used together to actually deploy the "StackForLambdaDeployment".
The CDK "CodePipeline Example" article obviously uses CodeBuild running on AWS. However, not charging time against the CodeBuild limit in Free Tier is definitely good for troubleshooting purposes. Therefore, let's run CodeBuild locally!
When trying something new, I like to get the "Hello World" version working completely before starting to incorporate it into my workflow. Said "Hello World" local CodeBuild, which uses a Java sample application, worked for me out of the box with just a few small exceptions.
First, the tutorial has you build the openjdk8 docker image, and gives the image location as ubuntu/java/openjdk-8. After you clone the aws-codebuild-docker-images, the appropriate directory, at the time of this article is actually ubuntu/unsupported_images/java/openjdk-8.
The second issue came in building the docker image, while running the stated "docker build -t aws/codebuild/java:openjdk-8 ." -- which requires an edit to the openjdk-8 dockerfile. I got an error about the jdk version being used not existing ("E: Version '8u171-b11-2~14.04' for 'openjdk-8-jdk' was not found"), so I just removed the specific version that it was looking for in the dockerfile, and everything worked fine... changing this ...
&& apt-get install -y openjdk-${JAVA_VERSION}-jdk=$JDK_VERSION \
to this...
&& apt-get install -y openjdk-${JAVA_VERSION}-jdk \
Now after getting the "hello world" local CodeBuild working, it's time to apply the same tactic to the "CodeBuild Example" CDK app we already have.
In the "CodeBuild Example" article, the buildspec for the Lambda CodeBuild is defined directly within the TypeScript, as follows (this is in the lib/pipeline-stack.ts file):
const lambdaBuild = new codebuild.PipelineProject(this, 'LambdaBuild', {
buildSpec: codebuild.BuildSpec.fromObject({
version: '0.2',
phases: {
install: {
commands: [
'cd lambda',
'npm install',
],
},
build: {
commands: 'npm run build',
},
},
artifacts: {
'base-directory': 'lambda',
files: [
'index.js',
'node_modules/**/*',
],
},
}),
environment: {
buildImage: codebuild.LinuxBuildImage.UBUNTU_14_04_NODEJS_10_14_1,
},
});
However, we actually want to extract that code into a file, which we can then utilize to locally run the CodeBuild. So first we create a buildspec file (I called it buildspec_pollyclient.yml in the repo) in the root repository directory, with the following contents. Note that to get the local CodeBuild working, a runtime must be supplied in the buildspec:
version: '0.2'
phases:
install:
commands:
- cd polly-client-lambda
- npm install
runtime-versions:
docker: 18
build:
commands:
- echo "Testing... building polly-client-lambda"
- npm run build
artifacts:
base-directory: polly-client-lambda
files:
- src/index.ts
- dist/index.js
- node_modules/**/*
Then, we need to update the lib/pipeline-stack.ts file to use the buildspec file we just created, so change the "const lambdaBuild" in the lib/pipeline-stack.ts file to pull the buildspec_pollyclient.yml file content, and then use it as the "buildSpec" value:
const lambdaCodeBuildBuildSpec = codebuild.BuildSpec.fromSourceFilename('./buildspec_pollyclient.yml')
const lambdaBuild = new codebuild.PipelineProject(this, 'LambdaBuild', {
buildSpec: lambdaCodeBuildBuildSpec,
environment: {
buildImage: codebuild.LinuxBuildImage.UBUNTU_14_04_NODEJS_10_14_1,
},
});
If we run "cdk deploy PipelineDeployingLambdaStack", the PipelineDeployingLambdaStack will reference the newly created buildspec file for the Lambda CodeBuild.
Now for the cool part. The codebuild_build.sh file, that we downloaded earlier to run CodeBuild locally, lets us specify a custom buildspec file as a command line argument. In the following command:
- "-c" means use AWS credentials from local machine
- "-i" specifies the build docker image
- "-a" is where on local to place code artifacts after they're build.
- "-s" is the source directory where the CDK app is (which also contains the Lambda directory and the buildspec file)
- "-b" is the magic that let's us specify the buildspec file to use when the CodeBuild docker runs
lambda_directory="<<add-local-path-where-this-git-repo-is-located"
./codebuild_build.sh -c -i aws/codebuild/standard:2.0 -a ~/artifacts -s $lambda_directory -b $lambda_directory/buildspec_pollyclient.yml
Now we have the ability to locally run the Lambda CodeBuild with the same buildspec file that's being used in the CodePipeline with AWS!
A final thing to note regarding CDK is that when deploying locally, we get the prompt stating "This deployment will make potentially sensitive changes according to your current security approval level... Do you wish to deploy these changes (y/n)?", in part because the stack is changing permissions. To prevent the app from asking this every time I deployed, this can be added to the cdk.json file in the root directory:
{
...
"requireApproval": "never"
}
The final change I made to the "CodePipeline Example" article is to setup TypeScript. I'm not going into any more detail in my article, but this article provided a great walk-through of adding the required dependencies and adding a tsconfig.json file to get TypeScript building.
It might seem like my article stitched together a few orthogonal topics. However at the end of the day, I wanted a NodeJS Lambda that was backed with TypeScript, deployed from a CodePipeline automatically when I checked in code, and had the functionality of allowing me to run CodeBuild on my local before hashing through build errors while charging time in the cloud. All have been accomplished.