This is the Fog driver for Chef Provisioning. It provides Amazon EC2, DigitalOcean, Google Compute Engine, IBM Softlayer, Joyent, OpenStack, Rackspace, vCloud Air and XenServer functionality.
These are the primary documents to help learn about using Provisioning and creating Provisioning drivers:
A note about key pairs - The key name used in fog_key_pair must be the same as the filename of the local key to be used. If the key does not exist in one of private_key_paths (which you can set in knife.rb or in a client.rb) it will be created.
You'll need to update your knife.rb to work:
You need one of the following for the driver:
fog:AWS:<account_id>:<region>
fog:AWS:<profile_name>
fog:AWS:<profile_name>:<region>
For example:
driver 'fog:AWS:<account_id>:<region>'
driver_options :compute_options => {
:aws_access_key_id => 'YOUR-ACCESS-KEY-ID',
:aws_secret_access_key => 'YOUR-SECRET-ACCESS-KEY',
:region => 'THE-REGION-YOU-WANT-TO-PROVISION-IN'
}For a full example see examples/aws/simple.rb.
Update your knife.rb to contain your DigitalOcean API token and the driver:
driver 'fog:DigitalOcean'
driver_options :compute_options => {
:digitalocean_token => 'token'
}For a full example see examples/digitalocean/simple.rb.
You'll need to update your knife.rb to work:
driver 'fog:Google'
driver_options :compute_options => {
:provider => 'google',
:google_project => 'YOUR-PROJECT-ID', # the name will work here
:google_client_email => 'YOUR-SERVICE-ACCOUNT-EMAIL',
:google_key_location => 'YOUR-SERVICE-P12-KEY-FILE-FULL-PATH.p12'
}In order to get the YOUR-SERVICE-P12-KEY-FILE.p12 you need to set up a Service
account. This is located at Home > Permissions > Service Accounts and you'll
need to create a new one to get a new key. After that place it some place such
as ~/.chef/ so chef-provisioning-fog can find it. Your google_client_email
would be something like: <UNIQUE_NAME>@<PROJECT>.iam.gserviceaccount.com.
For a full simple example see examples/google/simple.rb.
For an example that shows a different :disk_type see
examples/google/simple_different_disk.rb.
You'll need to update your knife.rb to work with this also:
driver 'fog:SoftLayer'
driver_options :compute_options => {
:provider => 'softlayer',
:softlayer_username => 'username',
:softlayer_api_key => 'api_key',
:softlayer_default_domain => 'example.com',
}Once you or your administrator has created a SoftLayer account you can manage your API key at https://control.softlayer.com/account/users
:bootstrap_options => {:key_name => 'label'} is looked up by_label; make sure
you have a public key created on control portal at
https://control.softlayer.com/devices/sshkeys with a matching label.
NOTE: the SoftLayer driver injects a custom post provisioning script that ensures some packages needed by chef-provisioning-fog to install chef are present (e.g. sudo). The injected script will call your :postInstallScriptUri if you define one. The driver will wait until the injected script is done. The driver and script communicate using userMetadata so you cannot use metadata.
For a full example see examples/softlayer/simple.rb.
You'll need to update your knife.rb to work:
driver 'fog:Joyent'
driver_options :compute_options => {
:joyent_username => 'YOUR-JOYENT-LOGIN',
:joyent_password => 'YOUR-JOYENT-PASSWORD',
:joyent_keyname => 'THE-SSH-KEY-YOUVE-UPLOADED',
:joyent_version => '7.3.0', # if you are using the joyent public cloud
:joyent_keyfile => 'YOUR-PRIVATE-SSH-KEY-LOCATION' # Such as '/Users/jasghar/.ssh/id_rsa'
}Tested this with the Joyent Public Cloud. For the package names, use the GUI to find the name(s) that you want to use. This is also required to figure out the Image UUID, there doesn't seem to be an effective way of doing this without the GUI.
For a more in-depth usage of this driver to use with either Private or Public Joyent cloud, checkout this blog post by mhicks from #smartos on freenode.
For a infrastructure container example see examples/joyent/infrastructure.rb.
You'll need to update your knife.rb to work:
driver 'fog:OpenStack'
driver_options :compute_options => {
:openstack_auth_url => 'http://YOUROPENSTACK-CLOUD:5000/v2.0/tokens',
:openstack_username => 'YOUR-USERNAME',
:openstack_api_key => 'YOUR-PASSWORD',
:openstack_tenant => 'YOUR-TENANT-ID-NAME'
}For a full example see examples/openstack/simple.rb.
For this example, you must configure knife.rb with your credentials and a region to operate on. This example is also available as a Github repo.
You must configure some credentials and region in a knife.rb file like so:
driver 'fog:Rackspace'
driver_options :compute_options => {
:rackspace_username => 'MY-RACKSPACE-USERr',
:rackspace_api_key => 'API-KEY-FOR-USER',
:rackspace_region => 'dfw' # could be 'org', 'iad', 'hkg', etc
}For a full example see examples/rackspace/simple.rb.
Docs TODO.
Docs TODO.
require 'chef/provisioning'
machine_batch do
machines search(:node, '*:*').map { |n| n.name }
action :destroy
endWhen you are done with the examples, run this to clean up:
$ chef-client -z destroy_all.rb
Chef Provisioning has two major abstractions: the machine resource, and drivers.
You declare what your machines do (recipes, tags, etc.) with the machine resource, the fundamental unit of Chef Provisioning. You will typically declare machine resources in a separate, OS/provisioning-independent file that declares the topology of your app--your machines and the recipes that will run on them.
The machine resources from the cluster.rb example are pretty straightforward. Here's a copy/paste, it'll create a database machine then one web server.
machine 'mario' do
recipe 'postgresql'
recipe 'mydb'
tag 'mydb_master'
end
num_webservers = 1
1.upto(num_webservers) do |i|
machine "luigi#{i}" do
recipe 'apache'
recipe 'mywebapp'
end
endYou will notice the dynamic nature of the number of web servers. It's all code, your imagination is the limit :)
Drivers handle the real work of getting those abstract definitions into real, physical form. They handle the following tasks, idempotently (you can run the resource again and again and it will only create the machine once--though it may notice things are wrong and fix them!):
- Acquiring machines from the cloud, creating containers or VMs, or grabbing bare metal
- Connecting to those machines via ssh, winrm, or other transports
- Bootstrapping chef onto the machines and converging the recipes you suggested
The driver API is separated out so that new drivers can be made with minimal effort (without having to rewrite ssh, tunneling, bootstrapping, and OS support). But to the user, they appear as a single thing, so that the machine acquisition can use its smarts to autodetect the other bits (transports, OS's, etc.).
Drivers save their data in the Chef node itself, so that they will be accessible to everyone who is using the Chef server to manage the nodes.
chef-provisioning-fog interacts with your cloud provider's endpoint. Most of
the time, default timeouts for the following would be sufficient.
Modify the defaults if your Fog endpoint takes awhile to send/receive API requests. Normally, if you see Excon timeouts you should tune these parameters.
with_driver 'fog:foo',
:compute_options => {
:connection_options => {
# set connection to persist (default is false)
:persistent => true,
# set longer connect_timeout (default is 60 seconds)
:connect_timeout => 360,
# set longer read_timeout (default is 60 seconds)
:read_timeout => 360,
# set longer write_timeout (default is 60 seconds)
:write_timeout => 360,
}
}Modify these timeouts if you need Chef to wait a bit of time to allow for the machine to be ready.
with_machine_options({
# set longer to wait for the instance to boot to ssh (defaults to 180)
:create_timeout => 360,
# set longer to wait for the instance to start (defaults to 180)
:start_timeout => 360,
# set longer to wait for ssh to be available if the instance is detected up (defaults to 20)
:ssh_timeout => 360
})Modify this if your chef client convergences take awhile.
with_machine_options({
:convergence_options => {
# set longer if you need more time to converge (default: 2 hours)
:chef_client_timeout => 120*60
}
})