ibmendoza/stretcher

Deployment tool with consul/serf event notification.

Stretcher

A deployment tool with Consul / Serf event.

Example manifest

src: s3://example.com/app.tar.gz
checksum: e0840daaa97cd2cf2175f9e5d133ffb3324a2b93
dest: /home/stretcher/app
commands:
  pre:
    - echo 'staring deploy'
  post:
    - echo 'deploy done'
  success:
    - echo 'deploy success'
  failure:
    - echo 'deploy failed!!'
    - cat >> /path/to/failure.log
excludes:
  - "*.pid"
  - "*.socket"

Run

stretcher agent

$ stretcher -h
Usage of stretcher:
  -random-delay float
        sleep [0,random-delay) sec on start
  -v	show version
  -version
        show version

with Consul

A stretcher agent is designed as running under "consul watch" and will be kicked by Consul event.

$ consul watch -type event -name deploy /path/to/stretcher

• -name: your deployment identity name.

with Serf

A stretcher agent can be running as Serf event handler.

$ serf agent -event-handler="user:deploy=/path/to/stretcher >> /path/to/stretcher.log 2>&1"

Load AWS credentials

When you specify a S3 URL in manifest, requires a AWS credential setting one of below.

• ~/.aws/config and ~/.aws/credentials (overridden by AWS_CONFIG_FILE environment variable.)
  • AWS_DEFAULT_PROFILE is supported to select a profile from multiple credentials in file.
• AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_DEFAULT_REGION environment variable.
• EC2 IAM role.
  • requires AWS_DEFAULT_REGION environment variable.

Deployment process

Preparing

This process is not included in a stretcher agent.

1. Create a tar(or tar.gz) archive for deployment.
2. Upload the archive file to remote server (S3 or HTTP(S)).
3. Create a manifest file (YAML) and upload it to remote server.

Executing with Consul

Create a consul event to kick stretcher agents.

$ consul event -name [event_name] [manifest_url]

$ consul event -name deploy s3://example.com/deploy-20141117-112233.yml

• -name: consul event name (specified by consul watch -name)

Executing with Serf

Create a serf user event to kick stretcher agents.

$ serf event [event_name] [manifest_url]

$ serf event deploy s3://example.com/deploy-20141117-112233.yml

• event_name: user event name (specified by serf event handler).

Executing as command

Stretcher can read a manifest URL from stdin simply.

$ echo s3://example.com/deploy-20141117-112233.yml | stretcher

You can execute stretcher via ssh or any other methods.

Deployment process

A stretcher agent executes a following process.

1. Receive a manifest URL as Consul/Serf event's payload.
2. Get a manifest.
3. Get src URL and store it to a temporary file, and Check checksum.
4. Invoke pre commands.
5. Extract src archive to a temporary directory.
6. Sync files from extracted archive to dest directory.

• use rsync -a --delete

7. Invoke post commands.

• Invoke success commands when the deployment process succeeded.
• Invoke failure commands when the deployment process failed.

Manifest spec

src

Source archive URL.

• URL schema: 's3', 'http', 'file'
• Format: 'tar', 'tar.gz'

src: http://example.com/src/archive.tar.gz

checksum

Checksum of source archive.

• Type: 'md5', 'sha1', 'sha256', 'sha512'

checksum: e0840daaa97cd2cf2175f9e5d133ffb3324a2b93

dest

Destination directory.

dest: /home/stretcher/app

dest_mode

Destination directory mode. Default: 0755

dest_mode: 0711

Destination directory mode will be set as...

1. src archive includes . => same as . in the archive.
2. src archive does not include . => dest_mode

commands

• pre: Commands which will be invoked at before src archive extracted.
• post: Commands which will be invoked at after dest directory synced.
• success: Commands which will be invoked at deployment process is succeeded.
• failure: Commands which will be invoked at deployment process is failed.

commands:
  pre:
    - echo 'staring deploy'
  post:
    - echo 'deploy done'
  success:
    - echo 'deploy success'
  failure:
    - echo 'deploy failed!!'
    - cat >> /path/to/failure.log

stretcher agent logs will be passed to STDIN of success and failure commands.

excludes

Pass to rsync --exclude arguments.

excludes:
  - "*.pid"
  - "*.socket"

exclude_from

Pass to rsync --exclude-from arguments. The file must be included in src archive.

exclude_from: exclude.list

Requirements

• tar
• rsync

tar and rsync must be exist in PATH environment.

If you use stretcher under systemd, You can see unfinished stdout with journald. You should add RateLimitBurst=0 into /etc/systemd/journald.conf for getting stdout completely.

LICENSE

The MIT License (MIT)

Copyright (c) 2014- FUJIWARA Shunichiro