kubernetes-ansible: A Ruby repository from heytrav

Install and configure Kubernetes on OpenStack using ansible and terraform

Main goals

Install etcd server
Install docker
Install flanneld and put config in etcd
Install Kubernetes package
Configure Kubernetes master
Configure Kubernetes minion
Install kube-dns service discovery and DNS resolution pod

Available Addons

DNS for Service Discovery
Kubernetes UI
Logging Service for containers

NOTE

Each addon is enabled by default but can be disabled by changing the options within group_vars/all.yml All of the addons depend on the DNS addon for Service Discovery.

Provision Openstack environment

Install terraform

  # change 0.6.9 to the desired version
  wget -q -O terraform.zip https://dl.bintray.com/mitchellh/terraform/terraform_0.6.9_linux_amd64.zip
  unzip terraform.zip -d /usr/local/bin

Install pip package (CentOS/RedHat)

  # yum install python-devel python-pip

Install the OpenStack command-line clients

When following the instructions in this section, replace PROJECT with the lowercase name of the client to install, such as nova. Repeat for each client. The following values are valid:

    - barbican - Key Manager Service API
    - ceilometer - Telemetry API
    - cinder - Block Storage API and extensions
    - glance - Image service API
    - heat - Orchestration API
    - magnum - Containers service API
    - manila - Shared file systems API
    - mistral - Workflow service API
    - murano - Application catalog API
    - neutron - Networking API
    - nova - Compute API and extensions
    - sahara - Data Processing API
    - swift - Object Storage API
    - trove - Database service API
    - tuskar - Deployment service API
    - openstack - Common OpenStack client supporting multiple services

    # How to install with pip:
    pip install python-PROJECTclient
    (Replace PROJECT with the lowercase name of the client)

    # How to update with pip:
    pip install --upgrade python-PROJECTclient
    (Replace PROJECT with the lowercase name of the client)

    # To remove the client, run the pip uninstall command:
    pip uninstall python-PROJECTclient
    (Replace PROJECT with the lowercase name of the client)

Additional OpenStack CLI information here

Download Openstack RC file from Openstack Project Web Interface(Access & Security --> API Access)
```
  source openrc.sh
  # prompted for your password for the Openstack Project
```

Provide configurations for Openstack (From GitHub branch https://github.com/CiscoCloud/kubernetes-ansible/tree/master/terraform)

  cp terraform/openstack.sample.tf openstack.tf
  cp terraform/terraform.tfvars terraform.tfvars

  # edit the terraform.tfvars file by providing the following

  - auth_url (found within *openrc.sh)
  - tenant_id (found within *openrc.sh)
  - tenant_name (found within *openrc.sh)
  - location of ssh public key and a unique name
  - VM Flavor for Master node (*nova flavor-list)
  - VM Flavor for Worker node (*nova flavor-list)
  - Network ID (*nova net-list)
  - OS Image Name (*nova image-list)
  - Number of worker nodes
  - size (GB) of storage for kubernetes master to use

  *nova - example if you using NOVA Compute API and extensions
  *openrc.sh - Openstack Project Web Interface

Note: You must use image with pre-installed cloud-init.

Provision Environment

  terraform get (Get modules)
  terraform plan (Checking configuration)
  terraform apply (Apply )

Verify SSH access to the hosts
```
  ansible -m ping all
```

Firewall notice

If you are running on a cloud provider make sure that firewall configuration permits traffic between nodes. If you used terraform to provision environment then a security group has already been created for you.

Port list on roles: TBD

Prepare environment

There is 3 main roles:

etcd server
kubernetes master
kubernetes node (minion)

You can safely combine etcd and kubernetes master on one host, eventually you can run kubernetes minion on that host also.

For this setup you will need 1 host that would be kubernetes master and 2 or more hosts as minions. At least 2 minion nodes are needed to use flannel or any other networking for kubernetes.

If you already have prepared hosts you can provide simple ansible inventory (sample is in root of project).

Run ansible playbooks

Use Getting started with Ansible if you are not familiar with ansible.

Verify that ansible can reach your hosts.

ansible -m ping all

Validate the global configurations found in group_vars/all.yml and update as needed.

To run ansible on hosts you prepared run:

ansible-playbook -i inventory setup.yml

If you used Terraform to provision your hosts, a plugin is provided that dynamically extracts the inventory from .tfstate file.

ansible-playbook setup.yml

The same plugin can be used to either print out a lists of hosts or to add those hosts to your local /etc/hosts file so that you can reference the hosts by name.

The following command will append the hosts to your /etc/hosts file.

./plugins/inventory/terraform.py --hostfile >> /etc/hosts

Validate Ansible playbooks

Install Serverspec environment :

bundle install --path vendor/bundle

Run Serverspec test for all nodes and specs in parallel (using 8 threads), print short summary in JSON format and provide 0 exit code for succeed validation of Ansible playbooks :

bundle exec rake -m -j 8

Run Serverspec tests for different plays in parallel :

bundle exec rake spec:play:All -m -j 8
bundle exec rake spec:play:Master -m -j 8
bundle exec rake spec:play:Node -m -j 8

Show all available Rake-tasks :

bundle exec rake -T

To use different RSpec output formats (json is default one) :

FORMAT=documentation bundle exec rake spec:play:All -m -j 8
FORMAT=json bundle exec rake spec:play:All -m -j 8
FORMAT=progress bundle exec rake spec:play:All -m -j 8

JSON output format

When using FORMAT=json (default) the output will contain tests summary only :

{
  "succeed": true,
  "example_count": 490,
  "failure_count": 0
}

Detailed results could be found inside serverspec_results.json file at project root directory :

[
  {
    "name": "docker::k-master-01",
    "exit_code": 0,
    "output": {
      "version": "3.4.0",
      "examples": [
        {
          "description": "should be installed",
          "full_description": "docker : Main | Package \"docker\" should be installed",
          "status": "passed",
          "file_path": "./roles/docker/spec/main_spec.rb",
          "line_number": 5,
          "run_time": 3.202775,
          "pending_message": null
        },
        {
          "description": "should be enabled",
          "full_description": "docker : Main | Service \"docker\" should be enabled",
          "status": "passed",
          "file_path": "./roles/docker/spec/main_spec.rb",
          "line_number": 9,
          "run_time": 0.443939,
          "pending_message": null
        }
      ],
      "summary": {
        "duration": 4.07774,
        "example_count": 3,
        "failure_count": 0,
        "pending_count": 0
      },
      "summary_line": "3 examples, 0 failures"
    }
  },
  {
    "name": "flannel::k-master-01",
    "exit_code": 0,
    "output": {
      "version": "3.4.0",
      "examples": [
        {
          "description": "should be installed",
          "full_description": "flannel : Main |  Service | Package \"flannel\" should be installed",
          "status": "passed",
          "file_path": "./roles/flannel/spec/main_spec.rb",
          "line_number": 6,
          "run_time": 3.253822,
          "pending_message": null
        }
      ],
      "summary": {
        "duration": 6.399068,
        "example_count": 10,
        "failure_count": 0,
        "pending_count": 0
      },
      "summary_line": "10 examples, 0 failures"
    }
  }
]

Check cluster deployment

Validate Control

Check if all nodes are ready
```
  kubectl get nodes -o wide
```
Check if all Pods, Replication Controllers and Services are running
```
  kubectl get rc,svc,po --all-namespaces -o wide
```

Check status of Kubernetes processes

  sudo systemctl status etcd kube-apiserver kube-controller-manager kube-scheduler -l

View logs of Kubernetes processes

  sudo journalctl -u etcd
  sudo journalctl -u kube-apiserver
  sudo journalctl -u kube-controller-manager
  sudo journalctl -u kube-scheduler

Verify DNS working

  # https://github.com/GoogleCloudPlatform/kubernetes/tree/v1.0.1/cluster/addons/dns
  # busybox.yaml
  apiVersion: v1
  kind: Pod
  metadata:
    name: busybox
    namespace: default
  spec:
    containers:
    - image: busybox
      command:
        - sleep
        - "3600"
      imagePullPolicy: IfNotPresent
      name: busybox
    restartPolicy: Always


  kubectl create -f busybox.yaml
  kubectl get pods busybox
  kubectl exec busybox -- nslookup kubernetes

Verify NAT settings
```
  sudo iptables -t nat -L -n -v
```

Validate Nodes