Cfer

Cfer is a lightweight toolkit for managing CloudFormation templates.

Read about Cfer here.

Support

Cfer is pre-1.0 software, and may contain bugs or incomplete features. Please see the license for disclaimers.

If you would like support or guidance on Cfer, or CloudFormation in general, I offer DevOps consulting services. Please Contact me and I'll be happy to discuss your needs.

You can also find me at @tilmonedwards. If you use Cfer, or are considering it, I'd love to hear from you.

Installation

Add this line to your application's Gemfile:

gem 'cfer'

And then execute:

$ bundle

Or install it yourself as:

$ gem install cfer

Usage

To quickly see Cfer in action, try converging the example stacks:

cfer converge vpc -t examples/vpc.rb --profile [YOUR-PROFILE] --region [YOUR-REGION]
cfer converge instance -t examples/instance.rb --profile [YOUR-PROFILE] --region [YOUR-REGION] --parameters KeyName:[YOUR-EC2-SSH-KEY]

You should see something like this:

Command line

Commands:
  cfer converge [OPTIONS] <stack-name>   # Creates or updates a cloudformation stack according to the template
  cfer generate [OPTIONS] <template.rb>  # Generates a CloudFormation template by evaluating a Cfer template
  cfer help [COMMAND]                    # Describe available commands or one specific command
  cfer tail <stack-name>                 # Follows stack events on standard output as they occur

Global options

--profile <profile>: The AWS profile to use (from your ~/.aws/credentials file)
--region <region>: The AWS region to use
--verbose: Also print debugging messages

`generate <template.rb>`

The generate subcommand evaluates the given Ruby script and prints the CloudFormation stack JSON to standard output.

The following options may be used with the generate command:

--no-pretty-print: Print minified JSON

`converge <stack-name>`

Creates or updates a CloudFormation stack according to the specified template.

The following options may be used with the converge command:

--follow (-f): Follows stack events on standard output as the create/update process takes place.
--stack-file <template.rb>: Reads this file from the filesystem, rather than the default <stack-name>.rb
--parameters <Key1>:<Value1> <Key2>:<Value2> ...: Specifies input parameters, which will be available to Ruby in the parameters hash, or to CloudFormation by using the Fn::ref function
--parameter-file <params_file.[yaml|json]: Specifies input parameters from a YAML or JSON file
--parameter-environment <env_name>: Requires --parameter-file. Merges the specified key in the YAML or JSON file into the root of the parameter file before passing it into the Cfer stack, i.e. to provide different constants for different AWS environments. The priority for parameters is, in ascending order, stack default, file, environment, and command line.
--on-failure <DELETE|ROLLBACK|DO_NOTHING>: Specifies the action to take when a stack creation fails. Has no effect if the stack already exists and is being updated.
--stack-policy <filename|URL|JSON string> (-s): Stack policy to apply to the stack in order to control updates; takes a local filename containing the policy, a URL to an S3 object, or a raw JSON string.
--stack-policy-during-update <filename|URL|JSON string> (-u): Stack policy as in --stack-policy option above, but applied as a temporary override to the permanent policy during stack update.
--s3-path <S3_PATH>: Path to an S3 bucket location where the template will be stored. This is required if the template output exceeds 51,200 bytes.
--force-s3: Forces Cfer to upload the template to S3, even if it's small enough to be uploaded directly to the cloudformation API.
--change <CHANGE_NAME>: Creates a CloudFormation Change Set, rather than immediately updating the stack.

`tail <stack-name>`

Prints the latest n stack events, and optionally follows events while a stack is converging.

The following options may be used with the tail command:

--follow (-f): Follows stack events on standard output as the create/update process takes place.
--number (-n): Print the last n stack events.

`remove <stack-name>`

Removes or deletes an existing CloudFormation stack.

cfer remove vpc --profile [YOUR-PROFILE] --region [YOUR-REGION]

This can also be done in the following way with the awscli tools, but now eliminates the need to install that package.

aws cloudformation delete-stack --stack-name vpc

Template Anatomy

See the examples directory for some examples of complete templates.

Parameters

Parameters may be defined using the parameter function:

parameter :ParameterName,
  type: 'String',
  default: 'ParameterValue'

Any parameter can be referenced either in Ruby by using the parameters hash:

parameters[:ParameterName]

Parameters can also be used in a CloudFormation reference by using the Fn::ref function:

Fn::ref(:ParameterName)

Resources

Resources may be defined using the resource function:

resource :ResourceName, 'AWS::CloudFormation::CustomResource', AttributeName: {:attribute_key => 'attribute_value'} do
  property_name 'property_value'
end

Gets transformed into the corresponding CloudFormation block:

"ResourceName": {
  "Type": "AWS::CloudFormation::CustomResource",
  "AttributeName": {
    "attribute_key": "attribute_value"
  },
  "Properties": {
    "PropertyName": "property_value"
  }
}

Outputs

Outputs may be defined using the output function:

output :OutputName, Fn::ref(:ResourceName)

Outputs may be retireved from other stacks anywhere in a template by using the lookup_output function.

lookup_output('stack_name', 'output_name')

Including code from multiple files

Templates can get pretty large, and splitting template code into multiple files can help keep things more manageable. The include_template function works in a similar way to ruby's require_relative, but within the context of the CloudFormation stack:

include_template 'ec2.rb'

You can also include multiple files in a single call:

include_template(
  'stack/ec2.rb',
  'stack/elb.rb'
)

The path to included files is relative to the base template file (e.g. the converge command -t option).

SDK

Embedding the Cfer SDK involves interacting with two components: The Client and the Stack. The Cfer Client is the interface with the Cloud provider.

Basic API

The simplest way to use Cfer from Ruby looks similar to the CLI:

  Cfer.converge! '<stack-name>', template: '<template-file>'

This is identical to running cfer converge <stack-name> --template <template-file>, but is better suited to embedding in Rakefiles, chef recipes, or your own Ruby scripts. See the Rakefile in this repository for how this might look.

Cfn Client

The Client is a wrapper around Amazon's CloudFormation client from the AWS Ruby SDK. Its purpose is to interact with the CloudFormation API.

Create a new client:

Cfer::Cfn::Client.new(stack_name: <stack_name>)

Cfer::Cfn::Client also accepts options to be passed into the internal Aws::CloudFormation::Client constructor.

`converge(stack)`

Creates or updates the CloudFormation stack to match the input stack object. See below for how to create Cfer stack objects.

client.converge(<stack>)

`tail(options = {})`

Yields to the specified block for each CloudFormation event that qualifies given the specified options.

client.tail number: 1, follow: true do |event|
  # Called for each CloudFormation event, as they occur, until the stack enters a COMPLETE or FAILED state.
end

Cfer Stacks

A Cfer stack represents a baked CloudFormation template, which is ready to be converted to JSON.

Create a new stack:

`stack_from_file`

stack = Cfer::stack_from_file(<file>, client: <client>)

`stack_from_block`

stack = Cfer::stack_from_block(client: <client>) do
  # Stack definition goes here
end

Contributing

This project uses git-flow. Please name branches and pull requests according to that convention.

Always use --no-ff when merging into develop or master.

This project also contains a Code of Conduct, which should be followed when submitting feedback or contributions to this project.

New features

Branch from develop
Merge into develop
Name branch feature/<feature-name>

Unreleased bugs

Branch from develop
Merge into develop
Name branch bugfix/<issue-id>

Bugfixes against releases

Branch from master
Merge into develop and master
Name branch hotfix/<issue-id>

Releases

Branch from develop
Merge into develop and master
Name branch release/<major.minor>

Release Notes

0.4.0

BREAKING CHANGES

Provisioning is removed from Cfer core and moved to cfer-provisioning

Enhancements

Adds support for assume-role authentication with MFA (see: https://docs.aws.amazon.com/cli/latest/userguide/cli-roles.html)
Adds support for yml-format parameter files with environment-specific sections.
Adds a DSL for IAM policies.
Adds cfer estimate command to estimate the cost of a template using the AWS CloudFormation cost estimation API.
Enhancements to chef provisioner to allow for references in chef attributes. (Thanks to @eropple)
Adds continue/rollback/quit selection when ^C is caught during a converge.
Stores Cfer version and Git repo information in the Repo metadata.
Added support for uploading templates to S3 with the --s3-path and --force-s3 options.
Added new way of extending resources, making plugins easier.
Added support for CloudFormation Change Sets via the --change option.

Bugfixes

0.3.0

Enhancements:

parameters hash now includes parameters that are set on the existing stack, but not passed in via CLI during a stack update.
parameters hash now includes defaults for parameters that were not passed on the CLI during a stack creation.
Adds a lookup_output function, for looking up outputs of stacks in the same account+region. (See #8)
Adds provisioning for cfn-init and chef-solo, including resource signaling.
Adds support for stack policies.
Cfer no longer validates parameters itself. CloudFormation will throw an error if something is wrong.
Adds release notes to the README.

Bugfixes:

Removes automatic parameter mapping in favor of an explicit function available to resources. (Fixes Issue #8)
No more double-printing the stack summary when converging a stack with tailing enabled.
Update demo to only use 2 AZs, since us-west-1 only has two.
AllowedValues attribute on parameters is now an array, not a CSV string. (Thanks to @rlister)

0.2.0

Enhancements:

Adds support for including other files via include_template function.
Adds basic Dockerfile

josephreynolds/cfer

Cfer

Support

Installation

Usage

Command line

Global options

generate <template.rb>

converge <stack-name>

tail <stack-name>

remove <stack-name>

Template Anatomy

Parameters

Resources

Outputs

Including code from multiple files

SDK

Basic API

Cfn Client

converge(stack)

tail(options = {})

Cfer Stacks

stack_from_file

stack_from_block

Contributing

New features

Unreleased bugs

Bugfixes against releases

Releases

Release Notes

0.4.0

BREAKING CHANGES

Enhancements

Bugfixes

0.3.0

Enhancements:

Bugfixes:

0.2.0

Enhancements:

`generate <template.rb>`

`converge <stack-name>`

`tail <stack-name>`

`remove <stack-name>`

`converge(stack)`

`tail(options = {})`

`stack_from_file`

`stack_from_block`