Turnkey Kubernetes cluster setup with Vagrant (1.7.2+) and CoreOS.
####If you're lazy, or in a hurry, jump to the TL;DR section.
- Vagrant
- a supported Vagrant hypervisor
- Virtualbox (the default)
- Parallels Desktop
- VMware Fusion or VMware Workstation
On MacOS X (and assuming you have homebrew already installed) run
brew update
brew install wget
The vagrant-winnfsd plugin will be installed in order to enable NFS shares.
Current Vagrantfile
will bootstrap one VM with everything needed to become a Kubernetes master and, by default, a couple VMs with everything needed to become Kubernetes minions.
You can change the number of minions and/or the Kubernetes version by setting environment variables NODES and KUBERNETES_VERSION, respectively. You can find more details below.
vagrant up
Kubernetes cluster is ready. but you need to set-up some environment variables that we have already provisioned for you. In the current terminal window, run:
source ~/.bash_profile
New terminal windows will have this set for you.
On Windows systems, kubectl
is installed on the master
node, in the /opt/bin
directory. To manage your Kubernetes cluster, ssh
into the master
node and run kubectl
from there.
vagrant ssh master
kubectl cluster-info
vagrant destroy
If you've set NODES
or any other variable when deploying, please make sure you set it in vagrant destroy
call above, like:
NODES=3 vagrant destroy
VirtualBox is the default hypervisor, and you'll probably need to disable its DHCP server
VBoxManage dhcpserver remove --netname HostInterfaceNetworking-vboxnet0
If you are using Parallels Desktop, you need to install vagrant-parallels provider
vagrant plugin install vagrant-parallels
Then just add --provider parallels
to the vagrant up
invocations above.
If you are using one of the VMware hypervisors you must buy the matching provider and, depending on your case, just add either --provider vmware-fusion
or --provider vmware-workstation
to the vagrant up
invocations above.
If you want to use Docker private repositories look for DOCKERCFG bellow.
Most aspects of your cluster setup can be customized with environment variables. Right now the available ones are:
-
NODES sets the number of nodes (minions).
Defaults to 2.
-
CHANNEL sets the default CoreOS channel to be used in the VMs.
Defaults to alpha.
While by convenience, we allow an user to optionally consume CoreOS' beta or stable channels please do note that as both Kubernetes and CoreOS are quickly evolving platforms we only expect our setup to behave reliably on top of CoreOS' alpha channel. So, before submitting a bug, either in this project, or in (Kubernetes or CoreOS) make sure it (also) happens in the (default) alpha channel 😄
-
COREOS_VERSION will set the specific CoreOS release (from the given channel) to be used.
Default is to use whatever is the latest one from the given channel.
-
SERIAL_LOGGING if set to true will allow logging from the VMs' serial console.
Defaults to false. Only use this if you really know what you are doing.
-
MASTER_MEM sets the master's VM memory.
Defaults to 1024 (in MB)
-
MASTER_CPUS sets the number os vCPUs to be used by the master's VM.
Defaults to 1.
-
NODE_MEM sets the worker nodes' (aka minions in Kubernetes lingo) VM memory.
Defaults to 2048 (in MB)
-
NODE_CPUS sets the number os vCPUs to be used by the minions's VMs.
Defaults to 1.
-
DOCKERCFG sets the location of your private docker repositories (and keys) configuration. However, this is only usable if you set USE_DOCKERCFG=true.
Defaults to "~/.dockercfg".
You can create/update a ~/.dockercfg file at any time by running
docker login <registry>.<domain>
. All nodes will get it automatically, at 'vagrant up', given any modification or update to that file. -
DOCKER_OPTIONS sets the additional
DOCKER_OPTS
for docker service on both master and the nodes. Useful for adding params such as--insecure-registry
. -
KUBERNETES_VERSION defines the specific kubernetes version being used.
Defaults to
1.0.6
. Versions prior to0.1.0
won't work with current cloud-config files. -
CLOUD_PROVIDER defines the specific cloud provider being used. This is useful, for instance, if you're relying on kubernetes to set load-balancers for your services.
Possible values are
gce
,gke
,aws
,azure
,vagrant
,vsphere
,libvirt-coreos
andjuju
.Defaults tobecause of kubernetes/kubernetes#9049.vagrant
,
So, in order to start, say, a Kubernetes cluster with 3 minion nodes, 4GB of RAM and 2 vCPUs per node one just would do...
NODE_MEM=4096 NODE_CPUS=2 NODES=3 vagrant up
Please do note that if you were using non default settings to startup your
cluster you must also use those exact settings when invoking
vagrant {up,ssh,status,destroy}
to communicate with any of the nodes in the cluster as otherwise
things may not behave as you'd expect.
You can automatically mount in your guest VMs, at startup, an arbitrary
number of local folders in your host machine by populating accordingly the
synced_folders.yaml
file in your Vagrantfile
directory. For each folder
you which to mount the allowed syntax is...
# the 'id' of this mount point. needs to be unique.
- name: foobar
# the host source directory to share with the guest(s).
source: /foo
# the path to mount ${source} above on guest(s)
destination: /bar
# the mount type. only NFS makes sense as, presently, we are not shipping
# hypervisor specific guest tools. defaults to `true`.
nfs: true
# additional options to pass to the mount command on the guest(s)
# if not set the Vagrant NFS defaults will be used.
mount_options: 'nolock,vers=3,udp,noatime'
# if the mount is enabled or disabled by default. default is `true`.
disabled: false
vagrant up
source ~/.bash_profile
This will start one master
and two minion
nodes, download Kubernetes binaries start all needed services.
A Docker mirror cache will be provisioned in the master
, to speed up container provisioning. This can take some time depending on your Internet connection speed.
Please do note that, at any time, you can change the number of minions
by setting the NODES
value in subsequent vagrant up
invocations.
Congratulations! You're now ready to use your Kubernetes cluster.
If you just want to test something simple, start with [Kubernetes examples] (https://github.com/GoogleCloudPlatform/kubernetes/blob/master/examples/).
For a more elaborate scenario [here] (https://github.com/pires/kubernetes-elasticsearch-cluster) you'll find all you need to get a scalable Elasticsearch cluster on top of Kubernetes in no time.
If you see something like this in the log:
==> master: Waiting for Kubernetes master to become ready...
error: unable to load file "temp/dns-controller.yaml": unable to connect to a server to handle "replicationcontrollers": couldn't read version from server: Get https://10.245.1.2/api: dial tcp 10.245.1.2:443: i/o timeout
error: no objects passed to create
You probably have a pre-existing Kubernetes config file on your system at ~/.kube/config
. Delete or move that file and try again.
This work is open source, and is licensed under the Apache License, Version 2.0.