Chef Cookbook Repository with a pocketknife Tutorial targeting Application Deployment

This is a Chef Cookbook repo that containts very simple examples of an application deployment that can be used with Matlux' pocketknife and Chef-solo, powered by Opscode Chef.

This repo can also be used with Chef-solo on its own.

This tutorial expects you to install the 'matlux' version of the Pocketknife tool to work. Appart from some aspects, most of the following can easily be adapted and used with the original version of the Igal's Pocketknife. However this tutorial will NOT WORK if chef-solo is ran as the root user. You have to use the --user option with a user different from root. This is a limitation that I'm looking to overcome in future versions of this cookbook repository.

Pre-requisites

Install the software on the machine you'll be running pocketknife on, this is a computer that will deploy configurations to other computers:

Install Ruby: http://www.ruby-lang.org/
Install Rubygems: http://rubygems.org/
Install chef-solo: http://wiki.opscode.com/display/chef/Installing+Chef+Client+and+Chef+Solo

Install Pocketknife ('matlux' version)

Install archive-tar-minitar: gem install archive-tar-minitar - pocketknife dependency.
Install rye: gem install rye - pocketknife dependency.
cd /path/of/your/choice
git clone git://github.com/matlux/pocketknife.git
export PATH=/path/of/your/choice/pocketknife:$PATH - add pocketknife to your PATH

1st Cookbook: the classic "Helloworld"

All you need to do is to define a new node using the chef JSON syntax for runlist and attributes. For example, to define a node with the hostname henrietta.swa.gov.it create the nodes/henrietta.swa.gov.it.json file, and add the contents below so it uses the myapp1 role and overrides its attributes to specify how many slots are available (not yet implemented in reality):

{
  "run_list": [
    "role[uat]",
    "recipe[helloworld]"
  ],
  "myapp": {
      "instanceNumber": 3
    }
}

Call the file henrietta.swa.gov.it inside the nodes directory` or your version of a [hostname.domain].

Operations on remote nodes will be performed using SSH. You should consider configuring ssh-agent so you don't have to keep typing in your passwords.

Finally, deploy your configuration to the remote machine and see the results. For example, lets deploy the above configuration to the henrietta.swa.gov.it host, which can be abbreviated as henrietta when calling pocketknife:

pocketknife --user bob henrietta

When deploying a configuration to a node, pocketknife will check whether Chef and its dependencies are installed. It something is missing, it will prompt you for whether you'd like to have it install them automatically.

To always install Chef and its dependencies when they're needed, without prompts, use the -i option, e.g. pocketknife -i henrietta. Or to never install Chef and its dependencies, use the -I option, which will cause the program to quit with an error rather than prompting if Chef or its dependencies aren't installed.

If something goes wrong while deploying the configuration, you can display verbose logging from pocketknife and Chef by using the -v option. For example, deploy the configuration to henrietta with verbose logging:

pocketknife --user bob -v henrietta

Once the program has executed you can verify that it has executed correctly by log into the remote computer and run the application as follow:

ssh henrietta.swa.gov.it
cd ~/chefwork/helloworld/bin
./start.sh

> Message is:
> Helloworld

There you are you're application is deployed and behaving correctly.

2nd Cookbook: an elastic 'myapp1' application

This application is elastic, it request 2 instances ("instanceReq") per node and assumes 3 slots are available ("instanceNumber").

You will see a myapp1 role is already defined inside the roles directory that describe common behavior and attributes of the "myapp" component using JSON syntax using chef's documentation.

Override cookbooks in the site-cookbooks directory. This has the same structure as cookbooks, but any files you put here will override the contents of cookbooks. This is useful for storing the original code of a third-party cookbook in cookbooks and putting your customizations in site-cookbooks.

Define roles in the roles directory that describe common behavior and attributes of your computers using JSON syntax using chef's documentation. For example, define a role called myapp1 by creating a file called roles/myapp1.json with this content:

{
  "name": "myapp1",
  "chef_type": "role",
  "json_class": "Chef::Role",
  "run_list": [
    "recipe[myapp]"
  ],
  "override_attributes": {
    "myapp": {
      "instanceReq": 2
    }
  }
}

{
  "run_list": [
    "role[uat]",
    "role[myapp1]"
  ],
  "myapp": {
      "instanceNumber": 3
    }
}

Call the file `henrietta.swa.gov.it` or your version of a [hostname.domain]. You will find an example in this project called `nodes/vr01.local.json` as an example.

What is interesting to understand is that myapp1 is a top level component that you are interested to install on a machine. myapp1 happens to use a commonly used sub-component called myapp which is a recipe. myapp has two sort of attributes (i.e. variables): those that are always true (closely bound to the myapp recipe) and those that are environmnet specific. Let's have a look at the chef-cookbooks-repo/cookbooks/myapp/attributes/default.rb file.

This is specific to UAT:

default["UAT"]["myapp"]["base_port"] = 8080
default["UAT"]["myapp"]["base_ssl_port"] = 8443

This is specific to DEV:

default["DEV"]["myapp"]["base_port"] = 8079
default["DEV"]["myapp"]["base_ssl_port"] = 8443

This is generic to any environment:

default["myapp"]["message"] = "hello world"
default["myapp"]["java_options"] = "-Xmx128M -Djava.awt.headless=true"
default["myapp"]["role_sub_dir"] = "myapp"

If a node uses the role uat it means that myapp recipe will use the UAT variables so environment and recipe are orthogonal to one another. It is useful to be able to re-use recipes across various environments without to have to rewrite the recipe. The benefit of this pattern is that once tested on a couple of environments it is easy to add an additional environment. Only the set of environment specific variable needs to be filled in to enable a new environment.

pocketknife --user bob henrietta

Once the program has executed you can verify that it has executed correctly by log into the remote computer and run the application as follow:

ssh henrietta.swa.gov.it
~/deploy/myapp/instance1/bin/start.sh

> Message is:
> Hi Helloworld

There you are you're application is deployed and behaving correctly.

If you really need to debug on the remote machine, you may be interested about some of the commands and paths:

chef-solo-apply (/tmp/usr/local/sbin/chef-solo-apply) will apply the configuration to the machine. You can specify -l debug to make it more verbose. Run it with -h for help.
csa (/tmp/usr/local/sbin/csa) is a shortcut for chef-solo-apply and accepts the same arguments.
~/chefwork/etc/chef/solo.rb contains the chef-solo configuration settings.
~/chefwork/etc/chef/node.json contains the node-specific configuration, like the runlist and attributes.
~/chefwork/var/local/pocketknife contains the cookbooks, site-cookbooks and roles describing your configuration.

Use pocketknife with a non root user

The matlux version of pocketknife does not require to run chef-solo as root. This tutorial assumes that you are running it as a non root user. You can run it as any user you like as long as its able to carry out what needs to be done in the recipe. This is ideal in a deployment scenario when the application developers often don't have root acces on the boxes they need to deploy onto.

Optional arguments:

--user myuser

--password passwordFile

Example:

pocketknife --user bob --password passwordExample.txt henrietta

The password file should contain a list of user and passwords as follow:

bob: bobpassword
user2: password2
user3: password3

Contributing

This software is published as open source at https://github.com/matlux/pocketknife

You can view and file issues for this software at https://github.com/matlux/pocketknife/issues

If you'd like to contribute code or documentation:

Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet
Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it
Fork the project
Start a feature/bugfix branch
Commit and push until you are happy with your contribution
Make sure to add tests for it. This is important so I don't break it in a future version unintentionally.
Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.
Submit a pull request using github, this makes it easy for me to incorporate your code.

matlux/chef-cookbooks-repo