Successfully automating your machines in the cloud using Ansible

Videos of Talk

Prerequisite Talks

Building an AWS Instance

Slow Walkthrough

Same material / Digestible pace / Longer Video:

9-Aug, 2014 Hands-On Training

Fast Overview

Same material / Quickly covered / Shorter Video:

26-June, 2014 Bay Area Python Interest Group (BayPIGgies)/Silicon Valley Python MeetUp Video

Practice Sessions

Hands-On Training:
- No sessions currently scheduled

Talk description

As we have seen with the previous talk, "Red Pill, Blue Pill Virtual Machines and Virtual Environments" ( GitHub / YouTube ), we can create virtual machines in the cloud.

But, how do you "stamp" those machines differently? If you need to build a web server, mail server, DNS server, and load balancer, each machine may have the same base image but needs to be configured differently.

If you manually configure those machines, what happens when you suddenly have a surge in traffic and need four more web servers? Or, what if one finds a vulnerability in a library like Heartbleed in OpenSSL as we recently encountered. A very safe option would be to rebuild these machines from scratch. If they were built manually, rebuilding these machines within minutes from scratch would be daunting, tedious and error prone.

There are several tools that have been built to fix this problem. Two of the most popular tools (Chef and Puppet ) are written in the Ruby programming language. And, especially for the most popular, Chef, one needs somewhat of a familiarity with that language to use the tool.

There are two more tools that are written in Python and are growing in popularity: Salt and Ansible. Ansible requires the least amount of set-up (if any) and has the simplest infrastructure (it simply uses commands over ssh like Fabric does). Ansible is the easiest tool to get started with if you are new in the machine build automation frameworks.

We will start with a newly built machine and obtain it's public IP address. We will configure the ansible_hosts file with the IP address, and add/build plays (like recipes) to gradually configure that machine so that it is a Django web server running in the cloud. When we are finished, we should have a running machine and a recipe to easily build a seconded machine with a few keystrokes.

P.S. If you haven't previously built an Amazon Web Instance, I highly recommend watching this video in advance of the talk.

Getting Started (Installing and following-along)

Clone this repo.

prompt> git clone https://github.com/glenjarvis/ansible_tutorial.git
Cloning into 'ansible_tutorial'...
remote: Reusing existing pack: 112, done.
remote: Total 112 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (112/112), 37.58 KiB | 0 bytes/s, done.
Resolving deltas: 100% (48/48), done.
Checking connectivity... done.

Change to the src directory.
```
prompt> cd ansible_tutorial
```

Configure the repo for your account and settings

prompt> ( cd src; python configure.py )

This script creates configuration files for using Ansible to
configure a newly-created virtual machine.
It has been tested by the author on an AWS free tier VM.
This has the best chance of working on an AWS free tier VM, or
failing that, on a VM with a recent version of CentOS.

    No configuration file found. Let me ask questions so that we can configure.

What is the path to your .pem key file for  the virtual machine?
--> ~/example_key.pem

What user to use to ssh to the remote system [ec2-user]?
-->
Configuring `ansible_hosts` file ./ansible_hosts...

What is the IP address of the virtual machine?
--> demos.glenjarvis.com

Configuration is complete.

Follow the examples (starting with the example_01 subdirectory).

Examples

Here is a list of the examples just in case there's any confusion in which order the examples should be executed:

( cd src; python configure.py ) (see instructions above)
( cd src/example_01; python access_machine.py )
( cd src/example_02; python auto_update_machine.py )
( cd src/example_03; less README.txt ) (Read the README.txt file; it's not really meant to be executed)
( cd src/example_04; ansible webservers -m ping; ansible webservers -vvv -m ping )
( cd src/example_05; ansible webservers -vvv -a 'sudo yum update -y' )
( cd src/playbook_examples; ansible-playbook demo_playbook_iter_01.yml )
( cd src/playbook_examples; ansible-playbook demo_playbook_iter_02.yml )
( cd src/playbook_examples; ansible-playbook demo_playbook_iter_03.yml )
( cd src/playbook_examples; ansible-playbook demo_playbook_iter_04.yml )
( cd src/playbook_examples; ansible-playbook demo_playbook_iter_05.yml )
( cd src/playbook_examples; ansible-playbook demo_playbook_iter_06.yml )
( cd src/playbook_examples; ansible-playbook demo_playbook_iter_07.yml )
( cd src/playbook_examples; ansible-playbook demo_playbook_iter_08.yml )
( cd src/playbook_examples; ansible-playbook demo_playbook_iter_09.yml )
( cd src/playbook_examples; less pedantically_commented_playbook ) (Read but don't execute: pedantically_commented_playbook.yml)
( cd src/role_examples; ansible-playbook demo_play_role_01.yml )
( cd src/role_examples; ansible-playbook demo_play_role_02.yml )
( cd src/role_examples; ansible-playbook demo_play_role_03.yml )
Exploration: Log into machine; sudo su - webuser; cd /home/webuser/sample_project; python manage.py runserver

Bio

Glen has been a full-time Python programmer since 2007 and has worked for companies such as IBM, UC Berkeley, Sprint, Informix, and many small start-ups. He has also worked both in the US and in the UK and has had Bioinformatics research published in Nucleic Acids Research (Oxford Journals). He is a certified DBA and has also been certified in Linux/Unix Administration.

He currently works for a start-up, RepairPal (for accurate car repair prices) using Ruby on Rails and Ansible. Additionally, he runs a small start up, Glen Jarvis, LLC, that does online technical training and assists employees obtaining telepresence in their current work place.

Glen is the organizer for the Silicon Valley Python MeetUp Group and an active member in the Bay Area Python Interest Group organization.

GitHub

Google+

Documentation for configure.py (if needed)

Although this probably won't be needed, here is the configure.py documentation that is used to help you configure your ansible.cfg and ansible_hosts files.

Help on module configure:

NAME
    configure - Configure the demo repository to make it easier to learn/follow

FILE
    ...ansible-tutorial/src/configure.py

DESCRIPTION
    This takes the machine name and the AWS pem key path and configures the
    ansible.cfg and ansible_hosts files. We also avoid the most common
    gotcha (permissions on the downloaded key file) by setting the
    permissions to be Read/Write for only the owner.

FUNCTIONS
    check_and_configure()
        Check/configure `ansible.cfg` and `ansible_hosts`
    
    configure_config()
        Assuming no ansible.cfg exists, ask questions and create a new one

    configure_hosts(hostfile)
        Assuming no hostfile file exists, ask questions and create a new one

    fix_pem_permissons(pem_file_path)
        Forcefully fix the PEM file permissions

        The largest problem that new users have when connecting ot their
        first AWS instance is that the permissions on the *.pem key that
        they downloaded is too permissive. It really isn't a *private* key
        if anyone else on the system (group or other) can read the file.
        Assuming no ansible.cfg exists, ask questions and create a new one

    configure_hosts(hostfile)
        Assuming no hostfile file exists, ask questions and create a new one
    
    fix_pem_permissons(pem_file_path)
        Forcefully fix the PEM file permissions
        
        The larest problem that new users have when connecting ot their
        first AWS instance is that the permissions on the *.pem key that
        they downloaded is too permissive. It really isn't a *private* key
        if anyone else on the system (group or other) can read the file.
        
        We change the permissions to the file given so that only the owner
        can read or write. There really isn't a reason to allow write
        permissions for the key, however, these are the default permissions
        for an ssh key.
    
    get_configured_hosts()
        Read the ansible.cfg file and parse hostfile pathname
    
    write_ansible_cfg_file(pem_file_path, user)
        Given a validated pem_file_path, write the ansible configuration file

DATA
    ANSIBLE_CONFIG_FILENAME = './ansible.cfg'
    ANSIBLE_CONFIG_FILEPATH = '...ansible-tutorial/src/ansible.cfg'
    ANSIBLE_HOSTS_FILENAME = './ansible_hosts'
    ANSIBLE_HOSTS_FILEPATH = '...ansible-tutorial/src/ansible_hosts'
    BASE_FILE = '...ansible-tutorial/src'

vikaschenny/LearnAnsible