Copier template for a CloudFormation Custom Resource Provider in Python

This copier template allows you to create a complete custom resource provider in minutes!

Out-of-the-box features include:

create the source for a custom cloudformation resource provider
support for semantic versioning your provider using git-release-tag
distribute the provider to buckets in all AWS regions in the world
re-recordable unit tests using botocore stubber recorder
deployable AWS Codebuild pipeline

getting started!

Let's say you want to create a custom resource for a Custom Domain of an AWS AppRunner service, because it does not yet exist. To create the project, type:

pip install copier
copier https://github.com/binxio/cloudformation-custom-provider-template /tmp/cfn-app-runner-custom-domain-provider

🎤 the name of your custom resource type?
   AppRunnerCustomDomain
🎤 The name of your resource provider project?
   cfn-app-runner-custom-domain-provider
🎤 The name of your Python module?
   cfn_app_runner_custom_domain_provider
🎤 a short description for the custom provider?
   manages app runner custom domains 
🎤 Python version to use
   3.9
🎤 Your full name?
   Mark van Holsteijn
🎤 Your email address?
   mark.vanholsteijn@xebia.com
🎤 the URL to git source repository?
   https://github.com/binxio/cfn-app-runner-custom-domain-provider
🎤 the AWS profile name
   integration-test
🎤 the AWS region name
   eu-central-1
🎤 prefix for the S3 bucket name to store the lambda zipfiles?
   binxio-public
🎤 Access to lambda zip files?
   public

> Running task 1 of 1: [[ ! -d .git ]] &&  ( git init &&  git add . &&  git commit -m 'initial import'  && git tag 0.0.0) || exit 0
Initialized empty Git repository in /tmp/cfn-app-runner-custom-domain-provider/.git/
[main (root-commit) b2ce863] initial import
 21 files changed, 619 insertions(+)
...

This creates a project with a working custom provider for the resource AppRunnerCustomDomain. Change to the directory and type make deploy-provider and make demo. Your provider will be up-and-running in less than 5 minutes!

what is in the box

When you type make help, you will get a list of all of available actions.

build                -  build the lambda zip file
fmt                  -  formats the source code
test                 -  run python unit tests
test-record          -  run python unit tests, while recording the boto3 calls
test-templates       -  validate CloudFormation templates

deploy               -  AWS lambda zipfile to bucket
deploy-all-regions   -  AWS lambda zipfiles to all regional buckets
undeploy-all-regions -  deletes AWS lambda zipfile of this release from all buckets in all regions

deploy-provider      -  deploys the custom provider
delete-provider      -  deletes the custom provider

deploy-pipeline      -  deploys the CI/CD deployment pipeline
delete-pipeline      -  deletes the CI/CD deployment pipeline

deploy-demo          -  deploys the demo stack
delete-demo          -  deletes the demo stack

tag-patch-release    -  create a tag for a new patch release
tag-minor-release    -  create a tag for a new minor release
tag-major-release    -  create a tag for new major release

run the unit tests

To run the unit tests, type:

$ pipenv install -d
$ make test

The unit test will test the scaffold implementation generated by the botocore stubber recorder. To create unit tests for your resource, edit the source code in ./tests/. To implement your custom resource, edit the source code under ./src/.

re-recordable unit tests

Once you have your custom resource provider, it undoubtedly does some AWS API calls. The botocore stubber recorder will allow you to create unit test by running the test against a real account. The tests will record the actual calls and generate the stubs. To run your unit tests, type:

$ make test-record

This will run the unit tests and record the AWS calls. To run the unit tests with the newly created stubs, type:

$ make test

The integration tests are run against the AWS profile and region you specified.

Deploy the zip file to the bucket

To copy the zip file with the source code of the AWS Lambda of the custom resource provider, type:

$ aws s3 mb s3://<bucket-prefix>-<bucket-region>

As you can see, the zipfile will be copied to a bucket name which consists of the prefix and the region name. This allows the zipfile to be made available for use in all regions.

Deploy the custom resource provider into the account

Now the zip file is available, you deploy the custom resource provider, by typing:

$ make deploy-provider

This deploys the provider as an AWS Lambda function. To configure the run-time parameters and permissions of the Lambda change the CloudFormation template in the directory ./cloudformation.

Deploy the custom resource demo

To deploy the demo CloudFormation stack using the custom resource provider, type:

$ make deploy-demo

This deploys an CloudFormation stack with an example custom resource as a CloudFormation stack. the run-time parameters and permissions of the Lambda change the CloudFormation template in the file ./cloudformation/demo.yaml. Change the configuration of the custom resource to match your implementation.

Version your custom resource provider

To version your custom resource provider, you can use the following commands:

make tag-patch-release    -  create a tag for a new patch release
make tag-minor-release    -  create a tag for a new minor release
make tag-major-release    -  create a tag for new major release

This will:

run the pre-tag command in the file ./release
commit all outstanding changes in the workspace
tag the commit with the new version.

To show the current version of the workspace, type:

make show-version

The utility git-release-tag implements this functionality.

Deploy provider to all regions

To deploy the current version of your provider to all regions, type:

make deploy-all-regions

This assumes you have buckets in all regions with the defined prefix.

Deploy CI/CD pipeline

To deploy the CI/CD pipeline based on AWS Codebuild, make sure that the AWS account can access the source repository. If that is the case, type:

make deploy-pipeline

Now every time you tag a new release, it will automatically be deployed to all regions.

conclusion

This copier template provides everything you need to quickly build, deploy and maintain a new custom AWS CloudFormation Provider!