sam-scaffold

A template for an AWS SAM project with continuous integration.

SAM is an extension of AWS's CloudFormation that makes it easier to define serverless applications. It is unopinionated on language choice, build tools, or project layout. This example project provides a set of opinions on those choices. It can be cloned and used as is to get a serverless project up and running quickly, or it can be used as a guideline for your own project.

This project features: TypeScript or ES2015 compilation, linting, unit testing, CloudFormation templates, continuous integration.

Included are two example lambda functions: helloWorld which does all the incredible things you expect from that name, and robots which implements a simple CRUD REST API using Cassava.

Source code

Two versions are provided: one with a TypeScript code base and another with JavaScript (ES2015 specifically). I highly recommend the TypeScript version, but the choice is yours.

Development

.
├── dev.sh
├── infrastructure
│   └── sam.yaml
└── src
    └── lambdas
        └── ...

The behaviour of a lambda function is determined by its source code (inside a subdirectory of src/lambdas) and the other serverless resources it has access to (inside the CloudFormation template infrastructure/sam.yaml).

Building

Compile the project with: npm run build.

Each lambda function will be built separately and packaged with its dependencies in a zip file in dist. For example src/lambdas/myfxn will be packaged in dist/myfxn/myfxn.zip. Don't worry about unnecessary libraries in node_modules being included. WebPack ensures only the source code referenced will be included.

Deployment

Deploying to a development account is easily done with the included script dev.sh. The script requires the aws cli installed and configured for a development account. For security reasons this should not be your production account. It also requires bash, which is a useful tool even on Windows.

Edit the top of dev.sh and replace STACK_NAME with a name that describes the project and replace BUILD_ARTIFACT_BUCKET with the name of an S3 bucket you have access to for build artifact storage.

These are the commands you can use...

./dev.sh build foo -- compile only the lambda function foo
./dev.sh deploy -- deploy the entire CloudFormation stack including all source code to the currently configured aws cli account.
./dev.sh upload foo -- only replace the the code for the lambda function foo.
./dev.sh invoke foo bar.json -- invoke and test the already deployed function foo with the input file bar.json.
./dev.sh delete -- delete the entire CloudFormation stack and all resources.

Linting

Linting is running a program that checks the source code for potential style and logical problems. The linter is set up to be run with: npm run lint.

Linting is provided by ESLint in JavaScript and TSLint in TypeScript. Check out their documentation for adjusting the rules to suit your preferred style.

Unit testing

Unit testing is provided by Mocha and Chai and is run with: npm run test.

Test files are located next to the file beign tested with .test added before the extension. eg: index.ts is beside test file index.test.ts. Just like libraries not referenced by index.ts WebPack will not include these file in the distribution.

Adding a new lambda function

Add a new directory inside src/lambdas named after your function. Inside there add a file index.ts if you're working in TypeScript or index.js if you're working in JavaScript. The file must have an export function handler that will be called by AWS.

Add a new AWS::Serverless::Function resource inside infrastructure/sam.yaml. Name it after your function with the first letter capitalized. Set the CodeUri to be the dist zip file that will be generated. eg: if your folder is src/lambdas/fooBar name your resource FooBarFunction with CodeUri: ../dist/fooBar/fooBar.zip.

Continuous Integration

.
├── buildspec.yml
└── infrastructure
    ├── ci.yaml
    ├── ciDockerImage
    │   ├── Dockerfile
    │   └── build.sh
    └── sam.yaml

Continuous integration is set up through another CloudFormation stack infrastructure/ci.yaml. This stack defines a CodePipeline that builds the project with CodeBuild, which runs the commands in buildspec.yml, and one of those commands deploys the SAM stack with CloudFormation. It's a CloudFormation stack that deploys another CloudFormation stack!

Again, for clarity: sam.yaml defines the SAM stack that is the definition of all your lambda functions and their resources; buildspec.yml defines your compile and deploy commands; ci.yaml defines the CI stack that watches for git repo changes and redeploys the SAM stack automatically.

The CI stack itself is not deployed automatically on changes. It must be deployed manually. This was chosen to increase the effort necessary to attack the account. The CI stack should rarely need to change. For help manually deploying a CloudFormation stack see the relevant AWS documentation.

Build secrets

You may run into a scenario where you need access to secrets during the build process. For example you have a private repository of packages and need an ssh key to access them.

The best way to handle these secrets is store them in an S3 bucket, give the CodeBuildServicePolicy permission to read that bucket, and then use aws cli commands to retrieve the secrets.

For example add this to ci.yaml:

# under CodeBuildServicePolicy.Properties.PolicyDocument.Statement
- Effect: Allow
  Action:
    - s3:GetObject
    - s3:ListBucket
  Resource:
    - !Sub "arn:aws:s3:::${MyBucketOfSecrets}"
    - !Sub "arn:aws:s3:::${MyBucketOfSecrets}/*"
  Principal:
    AWS: !GetAtt CiKeysAccessRole.Arn

# under CodeBuildProject.Properties.Environment.EnvironmentVariables
- Name: BUCKET_OF_SECRETS
  Value: !Ref MyBucketOfSecrets

and add this to buildspec.yml:

# under phases.install.commands
- aws s3 sync s3://BUCKET_OF_SECRETS/ ~/secrets

Setting up single stage CI

Single stage CI consists of only one CodePipeline. A single branch is watched for changes. When deploying for a single stage leave the GitHubBranchDest field empty.

The sequence of events goes like this:

a pull request is merged into the master branch
a git trigger causes CodePipeline to begin a release
CodeBuild fetches the release from GitHub
CodeBuild launches the build Docker image and runs the commands specified in buildspec.yml
the output artifacts are stored in S3
CloudFormation creates a change set for the SAM stack
a developer approves the changeset
CloudFormation executes the change set for the SAM stack

Setting up two stage CI

Two stage CI consists of two CodePipelines. The first CodePipeline watches a staging branch and deploys to a staging account. After successfully deploying and testing in staging the code is merged into a prod branch where the process repeats in production.