__ __
__ ______ ____ ___ ____ _/ /____ ____ ____/ /
/ / / / __ \/ __ `__ \/ __ `/ __/ _ \/ __ \/ __ /
/ /_/ / /_/ / / / / / / /_/ / /_/ __/ /_/ / /_/ /
\__, /\____/_/ /_/ /_/\__,_/\__/\___/\____/\__,_/
/____ matthewdavis.io, holla!
To get started simply clone the kubespray repo then you'll need to make a couple edits and you'll be on your way!
You'll want to use a linux based host to perform your deployment from and will need ansible installed locally (pip install ansible).
git clone https://github.com/kubernetes-incubator/kubespray
cd kubespray
cp -R inventory/sample inventory/mycluster
[all]
k8-master-01 ansible_host=5.0.2.104 ansible_port=22 ansible_connection=ssh ansible_user=centos
k8-node-01 ansible_host=5.0.2.65 ansible_port=22 ansible_connection=ssh ansible_user=centos
k8-node-02 ansible_host=5.0.2.66 ansible_port=22 ansible_connection=ssh ansible_user=centos
[kube-master]
k8-master-01
[kube-node]
k8-node-01
k8-node-02
[etcd]
k8-master-01
[k8s-cluster:children]
kube-node
kube-master
$ ansible all -i hosts.ini -m ping --become
k8-node-01 | SUCCESS => {
"changed": false,
"ping": "pong"
}
k8-master-01 | SUCCESS => {
"changed": false,
"ping": "pong"
}
k8-node-02 | SUCCESS => {
"changed": false,
"ping": "pong"
}
Make sure you have connectivity to your machines sorted out and you're ready to roll.
If all goes well, it shouldn't take but one command to fire things off. The entire provisioning process takes between 5-10 minutes generally so be patient and keep an eye on the ansible outputs.
$ ansible-playbook -i inventory/sample/hosts.ini cluster.yml -b -v
See Configurable Parameters in Kubespray
Now that we've deployed in a matter of minutes, here are some additional changes I highly recommend at a minimum:
| File | Property | Value |
|---|---|---|
| all.yml | kubelet_load_modules | true |
| k8s-cluster.yml | kube_version | v1.9.4 |
| k8s-cluster.yml | kube_api_pwd | something |
| k8s-cluster.yml | deploy_netchecker | true |
Install kubectl:
curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /bin/kubectl
Now query to make sure you can get to your resources:`
[centos@k8-master-01 ~]$ ./kubectl get node,pod,svc --all-namespaces -o wide
NAME STATUS ROLES AGE VERSION EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
no/k8-master-01 Ready master 1h v1.9.3+coreos.0 <none> CentOS Linux 7 (Core) 3.10.0-693.21.1.el7.x86_64 docker://17.12.1-ce
no/k8-node-01 Ready node 1h v1.9.3+coreos.0 <none> CentOS Linux 7 (Core) 3.10.0-693.21.1.el7.x86_64 docker://17.12.1-ce
no/k8-node-02 Ready node 1h v1.9.3+coreos.0 <none> CentOS Linux 7 (Core) 3.10.0-693.21.1.el7.x86_64 docker://17.12.1-ce
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE
kube-system po/calico-node-5zbkq 1/1 Running 0 1h 5.0.2.66 k8-node-02
kube-system po/calico-node-mlwrr 1/1 Running 0 1h 5.0.2.65 k8-node-01
kube-system po/calico-node-rm2d4 1/1 Running 1 1h 5.0.2.104 k8-master-01
kube-system po/kube-apiserver-k8-master-01 1/1 Running 1 1h 5.0.2.104 k8-master-01
kube-system po/kube-controller-manager-k8-master-01 1/1 Running 1 1h 5.0.2.104 k8-master-01
kube-system po/kube-dns-79d99cdcd5-bflc7 3/3 Running 0 1h 10.233.117.130 k8-node-01
kube-system po/kube-dns-79d99cdcd5-td4n8 3/3 Running 0 1h 10.233.65.65 k8-node-02
kube-system po/kube-proxy-k8-master-01 1/1 Running 1 1h 5.0.2.104 k8-master-01
kube-system po/kube-proxy-k8-node-01 1/1 Running 0 1h 5.0.2.65 k8-node-01
kube-system po/kube-proxy-k8-node-02 1/1 Running 0 1h 5.0.2.66 k8-node-02
kube-system po/kube-scheduler-k8-master-01 1/1 Running 1 1h 5.0.2.104 k8-master-01
kube-system po/kubedns-autoscaler-5564b5585f-fnv8n 1/1 Running 0 1h 10.233.117.129 k8-node-01
kube-system po/kubernetes-dashboard-69cb58d748-l97m6 1/1 Running 1 1h 10.233.116.194 k8-master-01
kube-system po/nginx-proxy-k8-node-01 1/1 Running 0 1h 5.0.2.65 k8-node-01
kube-system po/nginx-proxy-k8-node-02 1/1 Running 0 1h 5.0.2.66 k8-node-02
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
default svc/kubernetes ClusterIP 10.233.0.1 <none> 443/TCP 1h <none>
kube-system svc/kube-dns ClusterIP 10.233.0.3 <none> 53/UDP,53/TCP 1h k8s-app=kube-dns
kube-system svc/kubernetes-dashboard ClusterIP 10.233.31.36 <none> 443/TCP 1h k8s-app=kubernetes-dashboard
I've come across, and helped support, deploying to a multitude of environments finding that almost all documentation is missing these few nuggets that result in doom & gloom.
fatal: [rancher01]: FAILED! => {"msg": "The conditional check '{%- set certs = {'sync': False} -%}\n{% if gen_node_certs[inventory_hostname] or\n (not etcdcert_node.results[0].stat.exists|default(False)) or\n (not etcdcert_node.results[1].stat.exists|default(False)) or\n (etcdcert_node.results[1].stat.checksum|default('') != etcdcert_master.files|selectattr(\"path\", \"equalto\", etcdcert_node.results[1].stat.path)|map(attribute=\"checksum\")|first|default('')) -%}\n {%- set _ = certs.update({'sync': True}) -%}\n{% endif %}\n{{ certs.sync }}' failed. The error was: no test named 'equalto'\n\nThe error appears to have been in '/root/kubespray/roles/etcd/tasks/check_certs.yml': line 57, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: \"Check_certs | Set 'sync_certs' to true\"\n ^ here\n"}
pip install --upgrade Jinja2
If pip is not installed, install it via apt/yum install python-pip.
The documentation doesn't tell you that you'll need to have the python-netaddr package installed before provisioning. You'll end up with the following error:
TASK [kubespray-defaults : Configure defaults] ******************************************************************************************************************************************************************************************************************************************************************************
Monday 19 March 2018 02:29:37 +0000 (0:00:00.502) 0:00:08.900 **********
fatal: [localhost]: FAILED! => {"msg": "{u'no_proxy': u'{{ no_proxy }}', u'https_proxy': u\"{{ https_proxy| default ('') }}\", u'http_proxy': u\"{{ http_proxy| default ('') }}\"}: {%- if loadbalancer_apiserver is defined -%} {{ apiserver_loadbalancer_domain_name| default('') }}, {{ loadbalancer_apiserver.address | d
efault('') }}, {%- endif -%} {%- for item in (groups['k8s-cluster'] + groups['etcd'] + groups['calico-rr']|default([]))|unique -%} {{ hostvars[item]['access_ip'] | default(hostvars[item]['ip'] | default(hostvars[item]['ansible_default_ipv4']['address'])) }}, {%- if (item != hostvars[item]['ansible_hostname']) -%}
{{ hostvars[item]['ansible_hostname'] }}, {{ hostvars[item]['ansible_hostname'] }}.{{ dns_domain }}, {%- endif -%} {{ item }},{{ item }}.{{ dns_domain }}, {%- endfor -%} 127.0.0.1,localhost: The ipaddr filter requires python-netaddr be installed on the ansible controller"}
sudo apt-get install python-netaddr
yum install python-netaddr
TASK [kubernetes/preinstall : Stop if swap enabled] *************************************************************************************************************************************************************************************************************************************************************************
Monday 19 March 2018 02:32:51 +0000 (0:00:00.034) 0:00:11.243 **********
fatal: [localhost]: FAILED! => {
"assertion": "ansible_swaptotal_mb == 0",
"changed": false,
"evaluated_to": false
}
[yomateod@centos-1 kubespray]$ sudo swapoff -a
[yomateod@centos-1 kubespray]$ free -m
total used free shared buff/cache available
Mem: 7318 2541 3342 20 1434 4466
Swap: 0 0 0
You'll want to edit your /etc/fstab file on each node affected to make sure that the changes survive reboots.