jrnt30/kumogata

Kumogata is a tool for AWS CloudFormation. It can define a template in Ruby DSL.

Kumogata

Kumogata is a tool for AWS CloudFormation.

This is a format converter + useful tool. It supports the following format:

• JSON
• Ruby
• YAML
• JavaScript
• CoffeeScript (experimental)
• JSON5 (experimental)

It can define a template in Ruby DSL, such as:

AWSTemplateFormatVersion "2010-09-09"

Description (<<-EOS).undent
  Kumogata Sample Template
  You can use Here document!
EOS

Parameters do
  InstanceType do
    Default "t1.micro"
    Description "Instance Type"
    Type "String"
  end
end

Resources do
  myEC2Instance do
    Type "AWS::EC2::Instance"
    Properties do
      ImageId "ami-XXXXXXXX"
      InstanceType { Ref "InstanceType" }
      KeyName "your_key_name"

      UserData do
        Fn__Base64 (<<-EOS).undent
          #!/bin/bash
          yum install -y httpd
          service httpd start
        EOS
      end
    end
  end
end

Outputs do
  AZ do
    Value do
      Fn__GetAtt "myEC2Instance", "AvailabilityZone"
    end
  end
end

Ruby template structure is almost the same as JSON template.

(You can also use JSON templates)

Installation

$ gem install kumogata

Usage

Usage: kumogata <command> [args] [options]

Commands:
  create         PATH_OR_URL [STACK_NAME]   Create resources as specified in the template
  validate       PATH_OR_URL                Validate a specified template
  convert        PATH_OR_URL                Convert a template format
  update         PATH_OR_URL STACK_NAME     Update a stack as specified in the template
  delete         STACK_NAME                 Delete a specified stack
  list           [STACK_NAME]               List summary information for stacks
  export         STACK_NAME                 Export a template from a specified stack
  show-events    STACK_NAME                 Show events for a specified stack
  show-outputs   STACK_NAME                 Show outputs for a specified stack
  show-resources STACK_NAME                 Show resources for a specified stack
  diff           PATH_OR_URL1 PATH_OR_URL2  Compare templates logically (file, http://..., stack://...)

Options:
    -k, --access-key ACCESS_KEY
    -s, --secret-key SECRET_KEY
    -r, --region REGION
        --profile CONFIG_PROFILE
        --credentials-path PATH
        --config-path PATH
        --format TMPLATE_FORMAT
        --output-format FORMAT
        --skip-replace-underscore
        --deletion-policy-retain
    -p, --parameters KEY_VALUES
    -j, --json-parameters JSON
    -e, --encrypt-parameters KEYS
        --encryption-password PASS
        --skip-send-password
        --capabilities CAPABILITIES
        --disable-rollback
        --notify SNS_TOPICS
        --timeout MINUTES
        --result-log PATH
        --command-result-log PATH
        --detach
        --force
    -w, --ignore-all-space
        --color
        --no-color
        --debug
    -v, --verbose

KUMOGATA_OPTIONS

KUMOGATA_OPTIONS variable specifies default options.

e.g. KUMOGATA_OPTIONS='-e Password'

Create resources

$ kumogata create template.rb

If you want to save the stack, please specify the stack name:

$ kumogata create template.rb any_stack_name

If you want to pass parameters, please use -p option:

$ kumogata create template.rb -p "InstanceType=m1.large,KeyName=any_other_key"

Notice

The stack will be delete if you do not specify the stack name explicitly. (And only the resources will remain)

Convert JSON to Ruby

JSON template can be converted to Ruby template.

$ kumogata convert https://s3.amazonaws.com/cloudformation-templates-us-east-1/Drupal_Single_Instance.template

• Data that cannot be converted will be converted to Array and Hash
• :: is converted to __
  • Fn::GetAtt => Fn__GetAtt
• _{ ... } is convered to Hash
  • SecurityGroups [_{Ref "WebServerSecurityGroup"}] => {"SecurityGroups": [{"Ref": "WebServerSecurityGroup"}]}
• _path() creates Hash that has a key of path
  • _path("/etc/passwd-s3fs") { content "..." } => {"/etc/passwd-s3fs": {"content": "..."}}
• _user_data() creates Base64-encoded UserData
  • _user_data() has been removed
• _join() has been removed

String#fn_join()

Ruby templates will be converted as follows by String#fn_join():

UserData do
  Fn__Base64 (<<-EOS).fn_join
    #!/bin/bash
    /opt/aws/bin/cfn-init -s <%= Ref "AWS::StackName" %> -r myEC2Instance --region <%= Ref "AWS::Region" %>
  EOS
end

"UserData": {
  "Fn::Base64": {
    "Fn::Join": [
      "",
      [
        "#!/bin/bash\n",
        "/opt/aws/bin/cfn-init -s ",
        {
          "Ref": "AWS::StackName"
        },
        " -r myEC2Instance --region ",
        {
          "Ref": "AWS::Region"
        },
        "\n"
      ]
    ]
  }
}

Split a template file

• template.rb

Resources do
  _include 'template2.rb', :ami_id => 'ami-XXXXXXXX'
end

• template2.rb

myEC2Instance do
  Type "AWS::EC2::Instance"
  Properties do
    ImageId args[:ami_id]
    InstanceType { Ref "InstanceType" }
    KeyName "your_key_name"
  end
end

• Converted JSON template

{
  "Resources": {
    "myEC2Instance": {
      "Type": "AWS::EC2::Instance",
      "Properties": {
        "ImageId": "ami-XXXXXXXX",
        "InstanceType": {
          "Ref": "InstanceType"
        },
        "KeyName": "your_key_name"
      }
    }
  }
}

Encrypt parameters

• Command line

$ kumogata create template.rb -e 'Password1,Password2' -p 'Param1=xxx,Param2=xxx,Password1=xxx,Password2=xxx'

• Template

Parameters do
  Param1 { Type "String" }
  Param2 { Type "String" }
  Password1 { Type "String"; NoEcho true }
  Password2 { Type "String"; NoEcho true }
end # Parameters

Resources do
  myEC2Instance do
    Type "AWS::EC2::Instance"

    Properties do
      ImageId "ami-XXXXXXXX"

      UserData do
        Fn__Base64 (<<-EOS).fn_join
          #!/bin/bash
          /opt/aws/bin/cfn-init -s <%= Ref "AWS::StackName" %> -r myEC2Instance --region <%= Ref "AWS::Region" %>
        EOS
      end
    end

    Metadata do
      AWS__CloudFormation__Init do
        config do
          commands do
            any_command do
              command (<<-EOS).fn_join
                ENCRYPTION_PASSWORD="`echo '<%= Ref Kumogata::ENCRYPTION_PASSWORD %>' | base64 -d`"

                # Decrypt Password1
                echo '<%= Ref "Password1" %>' | base64 -d | openssl enc -d -aes256 -pass pass:"$ENCRYPTION_PASSWORD" > password1

                # Decrypt Password2
                echo '<%= Ref "Password2" %>' | base64 -d | openssl enc -d -aes256 -pass pass:"$ENCRYPTION_PASSWORD" > password2
              EOS
            end
          end
        end
      end
    end
  end # myEC2Instance
end # Resources

Iteration

You can use the Iteration in the template using _(...) method.

Resources do
  ['instance1', 'instance2', 'instance3'].echo {|instance_name|
    _(instance_name) do
      Type "AWS::EC2::Instance"
      Properties do
        ImageId "ami-XXXXXXXX"
        InstanceType { Ref "InstanceType" }
        KeyName "your_key_name"

        UserData (<<-EOS).undent.encode64
          #!/bin/bash
          yum install -y httpd
          service httpd start
          hostname #{instance_name}
        EOS
      end
    end
  }
end

Post command

You can run shell/ssh commands after building servers using _post().

• Template

Parameters do
  ...
end

Resources do
  ...
end

Outputs do
  MyPublicIp do
    Value { Fn__GetAtt name, "PublicIp" }
  end
end

_post do
  my_shell_command do
    command <<-EOS
      echo <%= Key "MyPublicIp" %>
    EOS
  end
  my_ssh_command do
    ssh do
      host { Key "MyPublicIp" } # or '<%= Key "MyPublicIp" %>'
      user "ec2-user"
      # see http://net-ssh.github.io/net-ssh/classes/Net/SSH.html#method-c-start
      #options :timeout => 300
      #connect_tries 36
      #retry_interval 5
      #request_pty true
    end
    command <<-EOS
      hostname
    EOS
  end
end

• Execution result

...
Command: my_shell_command
Status: 0
1> 54.199.251.30

Command: my_ssh_command
Status: 0
1> ip-10-0-129-20

(Save to `/foo/bar/command_result.json`)

JavaScript template

You can also use the JavaScript template instead of JSON and Ruby.

function fetch_ami() {
  return "ami-XXXXXXXX";
}

/* For JS Object is evaluated last, it must be enclosed in parentheses */
({
  Resources: { /* comment */
    myEC2Instance: {
      Type: "AWS::EC2::Instance",
      Properties: {
        ImageId: fetch_ami(),
        InstanceType: "t1.micro"
      }
    }
  },
  Outputs: {
    AZ: { /* comment */
      Value: {
        "Fn::GetAtt": [
          "myEC2Instance",
          "AvailabilityZone"
        ]
      }
    }
  }
})

/*
 {
   "Resources": {
     "myEC2Instance": {
       "Type": "AWS::EC2::Instance",
       "Properties": {
         "ImageId": "ami-XXXXXXXX",
         "InstanceType": "t1.micro"
       }
     }
   },
   "Outputs": {
     "AZ": {
       "Value": {
         "Fn::GetAtt": [
           "myEC2Instance",
           "AvailabilityZone"
         ]
       }
     }
   }
 }
 */

Convert JSON template to JavaScript

$ kumogata convert Drupal_Single_Instance.template --output-format=js

CoffeeScript template

You can also use the CoffeeScript template instead of JSON and Ruby.

fetch_ami = () -> "ami-XXXXXXXX"

/* For JS Object is evaluated last, it must use `return` */
return {
  Resources:
    myEC2Instance:
      Type: "AWS::EC2::Instance",
      Properties:
        ImageId: fetch_ami(),
        InstanceType: "t1.micro"
  Outputs:
    AZ: # comment
      Value:
        "Fn::GetAtt": [
          "myEC2Instance",
          "AvailabilityZone"
        ]
}

###
 {
   "Resources": {
     "myEC2Instance": {
       "Type": "AWS::EC2::Instance",
       "Properties": {
         "ImageId": "ami-XXXXXXXX",
         "InstanceType": "t1.micro"
       }
     }
   },
   "Outputs": {
     "AZ": {
       "Value": {
         "Fn::GetAtt": [
           "myEC2Instance",
           "AvailabilityZone"
         ]
       }
     }
   }
 }
###

YAML template

You can also use the YAML template instead of JSON and Ruby.

---
Resources:
  myEC2Instance:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: ami-XXXXXXXX
      InstanceType: t1.micro
Outputs:
  AZ:
    Value:
      Fn::GetAtt:
      - myEC2Instance
      - AvailabilityZone

# {
#   "Resources": {
#     "myEC2Instance": {
#       "Type": "AWS::EC2::Instance",
#       "Properties": {
#         "ImageId": "ami-XXXXXXXX",
#         "InstanceType": "t1.micro"
#       }
#     }
#   },
#   "Outputs": {
#     "AZ": {
#       "Value": {
#         "Fn::GetAtt": [
#           "myEC2Instance",
#           "AvailabilityZone"
#         ]
#       }
#     }
#   }
# }

Convert JSON template to YAML

$ kumogata convert Drupal_Single_Instance.template --output-format=yaml

JSON5 template

You can also use the JSON5 template instead of JSON and Ruby.

{
  Resources: { /* comment */
    myEC2Instance: {
      Type: "AWS::EC2::Instance",
      Properties: {
        ImageId: "ami-XXXXXXXX",
        InstanceType: "t1.micro"
      }
    }
  },
  Outputs: {
    AZ: { /* comment */
      Value: {
        "Fn::GetAtt": [
          "myEC2Instance",
          "AvailabilityZone"
        ]
      }
    }
  }
}

/*
 {
   "Resources": {
     "myEC2Instance": {
       "Type": "AWS::EC2::Instance",
       "Properties": {
         "ImageId": "ami-XXXXXXXX",
         "InstanceType": "t1.micro"
       }
     }
   },
   "Outputs": {
     "AZ": {
       "Value": {
         "Fn::GetAtt": [
           "myEC2Instance",
           "AvailabilityZone"
         ]
       }
     }
   }
 }
 */

Outputs Filter

Outputs do
  MyPublicIp do
    Value { Fn__GetAtt "MyInstance", "PublicIp" }
  end
end

_outputs_filter do |output|
  outputs["MyPublicIp"].gsub!('.', '_')
  # MyPublicIp: XXX.XXX.XXX.XXX => XXX-XXX-XXX-XXX
end

_post do
  ...
end

Configuration File

Kumogata supports aws-sdk configuration file.

[default]
aws_access_key_id=AKIAIOSFODNN7EXAMPLE
aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
aws_session_token=texample123324

Demo

• Create resources
  • https://asciinema.org/a/7979
• Convert a template
  • https://asciinema.org/a/7980
• Create a stack while outputting the event log
  • https://asciinema.org/a/8075
• Create a stack and run post commands
  • https://asciinema.org/a/8088

Similar tools

• Codenize.tools