A Gradle Plugin to assist in the initialization of a NetflixOSS-based cloud and continuous delivery environment.
$ git clone git@github.com:Netflix-Skunkworks/zerotocloud-gradle.git
$ cd zerotocloud-gradle
$ ./gradlew clean build install
Follow this guide.
$ export AWS_ACCESS_KEY_ID=<ACCESS_KEY_ID>
$ export AWS_SECRET_KEY=<SECRET_KEY_ID>
$ cd example/
$ ./gradlew initCloud
Go get a coffee... Things are happening now.
During initialization, the plugin is setting your account up in a NetflixOSS-friendly way.
- It is creating a new SSH key pair, which it will download to your
~/.ssh/
directory; - It is creating a Base IAM Role, which will be applied to instances that are deployed through the plugin;
- It is creating a Full Privilege IAM Role, which will be applied to those instances needing more access (like Asgard);
- It is creating a
base
security group, which will be applied to deployments performed through the plugin;- exposes ports 22 and 8080
- In this order, it will:
- Create a new instance for the Bakery; download and install aminator on that instance; and tag the instance for identification later;
- Create a new instance for Eureka; configure eureka for deployment; download and install eureka on that instance; and tag the instance for identification later;
- Create a new instance for Asgard; configure asgard for deployment; download and install asgard on that instance; and tag the instance for identification later;
- For each of these, it will also create a respective security group
The Bakery, Eureka, and Asgard are all components that will be used in a continuous delivery workflow. The plugin will provide some "glue" between those components to enable you to perform truly continuous delivery, all from a Gradle build.
Similar to the example above, where you initialized your AWS account and setup the Bakery, Eureka, and Asgard, by just applying the plugin to a vanilla Gradle build script, you can similarly apply the plugin to your project's build script to leverage the baking and deployment services that are contained internally.
Consider the following Gradle build, which is a Ratpack web application that we want to get deployed to the cloud:
buildscript {
repositories {
jcenter()
mavenLocal()
}
dependencies {
classpath 'io.ratpack:ratpack-gradle:0.9.11'
classpath 'com.netflix.nebula:nebula-ospackage-plugin:2.0.+'
classpath 'com.netflix.gradle.plugins:cloud:0.1-SNAPSHOT'
}
}
apply plugin: 'idea'
apply plugin: 'io.ratpack.ratpack-groovy'
apply plugin: 'nebula-ospackage-application'
apply plugin: 'nebula-ospackage-daemon'
apply plugin: 'com.netflix.cloud'
repositories {
jcenter()
}
dependencies {
compile 'com.netflix.eureka:eureka-client:1.1.145'
runtime 'org.apache.logging.log4j:log4j-slf4j-impl:2.0.1'
runtime 'org.apache.logging.log4j:log4j-api:2.0.1'
runtime 'org.apache.logging.log4j:log4j-core:2.0.1'
runtime 'com.lmax:disruptor:3.3.0'
}
ospackage {
from("${buildDir}/install") {
into "/apps/${project.name}"
}
requires("openjdk-7-jdk")
// will override to Java 8
preInstall file("scripts/preInstall.sh")
}
daemon {
daemonName = "${project.name}"
command = "/apps/${project.name}/bin/${project.name}"
}
task bake << {
def amiId = bakery.bakeryService.bake("${bakery.pubDir}/${project.tasks.buildDeb.archivePath.name}")
asgard.asgardService.lastAmi = amiId
}
task deploy << {
asgard.asgardService.deployBuilder()
.app("eutest")
.region("us-east-1")
.capacity(1, 1, 1)
.zones(["us-east-1b"])
.keyName(cloud.keyPairName)
.securityGroups([cloud.baseSecurityGroup])
.instanceType('m1.small')
.instanceProfile(cloud.instanceProfileName)
.build().deploy()
}
project.tasks.buildDeb.dependsOn = ['installApp']
project.tasks.publishDebs.dependsOn = ['buildDeb']
project.tasks.bake.dependsOn = ['publishDebs']
project.tasks.deploy.dependsOn = ['bake']
This build script will utilize the Nebula OS Package Plugin to generate a .deb file, which will be the OS package that will be baked into our server group image.
The bake
task utilizes the BakeryService
to:
- publish the os package to the Bakery server;
- invoke aminator on the Bakery server to create an AMI for distribution;
- once the AMI is created, its ID will be placed in state on the
AsgardService
, which will be used for deployment
The deploy
task has been programmatically added, which will utilize the AsgardService
's fluent deployment API to deploy
our application. In the case where a server group already exists, a Red/Black deployment
will be performed, allowing us to easily roll-back.
The lines at the bottom of the script ensure that the tasks are executed in proper order.
Every aspect of the plugin and its contained services is configurable using convention mappings:
cloud {
iamRole = "BaseIAMRole"
instanceProfileName = "BaseIAMRole_InstanceProfile"
keyPairName = "nf-oss"
assumeRolePolicyDoc = getClass().getResourceAsStream("/assumeRolePolicy.json").text
iamPolicyDoc = getClass().getResourceAsStream("/baseIamPolicy.json").text
region = "us-east-1"
remoteUser = "ubuntu"
baseSecurityGroup = "base"
baseAmiArch = "amd64"
baseAmiHypervisor = "xen"
baseAmiOwner = "099720109477" /* Canonical */
baseAmiName = "ubuntu/images/ebs/ubuntu-trusty-14.04-amd64-server" /* We'll lookup the latest one... */
baseAmiIsPublic = true
sshKey = new File(new File(System.properties['user.home'], '.ssh'), keyPairName)
}
bakery {
// all of "cloud", plus:
name = "bakery"
instanceType = "m3.medium"
securityGroup = "bakery"
checkPort = 22 /* informs the initializer to wait for this port to be available before considering success */
bakeryEnvironment = "ec2_aptitude_linux"
pubDir = "/repo" /* this is the directory where .deb files will be "published */
}
eureka {
// all of "cloud", plus:
instanceType = "m3.medium"
securityGroup = "eureka"
checkPort = 8080
availabilityZones = ['us-east-1a', 'us-east-1b', 'us-east-1c']
}
asgard {
// all of "cloud", except these are overridden:
iamPolicyDoc = getClass().getResourceAsStream("/fullPrivilegeIamPolicy.json").text
iamRole = "AsgardIAMRole"
instanceProfileName = "AsgardIAMRole_InstanceProfile"
// add't config:
instanceType = "m3.large"
securityGroup = "asgard"
checkPort = 8080
}
Dan Woods
See LICENSE.txt