/tendrl-vagrant

Vagrant setup for a tendrl-managed cluster

Tendrl/api development with Vagrant

A Vagrant setup for development of tendrl-api.

This will setup as many gluster storage nodes as you want with a number of bricks that you can define! It works with your local tendrl-api server and tendrl-ui setup. It uses vagrant's ansible plugin, so you will need ansible (>=2.4) on your host machine.

Requirements

  • macOS with Virtualbox (starting 5.1.30) or
  • RHEL 7.4/Fedora 27 with KVM/libvirt
  • Ansible (starting 2.4.0.0)
  • Vagrant (starting 1.9.1)
  • git

Installation instructions for Vagrant / Ansible

On RHEL 7.4

  • make sure you are logged in as a user with sudo privileges
  • make sure your system has the following repositories enabled (yum repolist)
    • rhel-7-server-rpms
    • rhel-7-server-extras-rpms
  • install the requirements
    • sudo yum groupinstall "Virtualization Host"
    • sudo yum install ansible git gcc libvirt-devel
    • sudo yum install https://releases.hashicorp.com/vagrant/2.0.1/vagrant_2.0.1_x86_64.rpm
  • start libvirtd
    • sudo systemctl enable libvirtd
    • sudo systemctl start libvirtd
  • enable libvirt access for your current user
    • sudo gpasswd -a $USER libvirt
  • as your normal user, install the libvirt plugin for vagrant
    • vagrant plugin install vagrant-libvirt

On Fedora 27

  • make sure you are logged in as a user with sudo privileges
  • make sure your system has the following repositories enabled (dnf repolist)
    • fedora
    • fedora-updates
  • install the requirements
    • sudo dnf install ansible git gcc libvirt-devel libvirt qemu-kvm
    • sudo dnf install vagrant vagrant-libvirt
  • start libvirtd
    • sudo systemctl enable libvirtd
    • sudo systemctl start libvirtd
  • enable libvirt access for your current user
    • sudo gpasswd -a $USER libvirt
  • as your normal user, install the libvirt plugin for vagrant
    • vagrant plugin install vagrant-libvirt

On macOS High Sierra

  • install the requirements

Get started

  • Clone this repository
    • git clone https://github.com/shirshendu/tendrl-vagrant.git
  • Goto the folder in which you cloned this repo
    • cd tendrl-vagrant
  • if you are a returning user run git pull to ensure you have the latest updates
  • if you are on RHEL/Fedora and your don't want your libvirt storage domain default to be used, override the storage domain like this
    • export LIBVIRT_STORAGE_POOL=images
  • cp tendrl.conf.yml.sample tendrl.conf.yml
    • Decide how many storage nodes and how many bricks you need
    • Decide if you want vagrant to initialize the cluster (gdeploy) for you
    • If you opted to initialize the cluster, decide whether you want to deploy tendrl
    • edit options in this file
  • Run vagrant up
    • Wait a while

Usage

  • Always make sure you are in the git repo - vagrant only works in there!
  • After vagrant up you can connect to each VM with vagrant ssh and the name of the VM you want to connect to
  • The single central server VM is called tendrl-server
  • Each storage node VM is called tendrl-node-x where x starts with 1
    • tendrl-node-1 is your first VM and it counts up depending on the amount of VMs you spawn
  • If you get the following error:

Error while activating network: Call to virNetworkCreate failed: error creating bridge interface virbr0: File exists.

Please try restarting libvirtd with sudo systemctl restart libvirtd

  • There are also other vagrant commands you should check out!
    • if you want to throw away everything: vagrant destroy -f
    • if you want to freeze the VMs and continue later: vagrant suspend
    • Try vagrant -h to find out about them
    • if you run vagrant up again you without running vagrant destroy before you will overwrite your configuration and vagrant may loose track of some VMs (it's safe to remove them manually)
  • modify the VMMEM and VMCPU variables in the Vagrant file to change the VM resources, adjust VMDISK to change brick device sizes

What happens under the covers

  • After starting the storage node VMs:
    • the hosts file is prepopulated
    • all glusters packages are pre-installed (allows you to continue offline)
    • the centos images are subscribed to epel, copr repositories
  • If you decided to have vagrant initialize the cluster
    • a gdeploy.conf is generated
    • gdeploy was executed with the gdeploy.conf file
    • cluster is peered
    • all block devices have been set up of VGs, LVs, formatted and mounted (gdeploy's standard backend-setup)
    • brick directories have been created
  • If you decided to deploy tendrl
    • tendrl-ansible roles are downloaded
    • an additional VM will run tendrl server components
    • the Ansible inventory and tendrl install playbook have been generated
    • the installation playbook has been executed on the Tendrl server and storage nodes
    • the Tendrl UI is reachable on the IP address of the eth1 adapter of the Tendrl VM (the URL also displayed after the installer finished)

Clean up / Refresh images

If you like to clean up disk space or there are updates to the images do the following:

  • on VirtualBox - remove the VM instances named packer-tendrl-server-... and packer-tendrl-node-... (these are base images for the clones)
  • on libvirt
    • run virsh vol-list default to list all images in your default storage pool (adjust the name if you are using a different one)
    • run virsh vol-delete tendrl-node-... default and virsh vol-delete tendrl-server-... default to delete the images starting with tendrl-node-... and tendrl-server-... (replace with full name) from the default pool
  • run vagrant box update

Next time you do vagrant up it will automatically pull new images.

Known issues

  • vagrant up overrides your state - if there are still VMs running and you do vagrant up it will override the vagrant_env.conf and vagrant will loose track of your existing VMs
    • try to remember to run vagrant destroy -f before you do another vagrant up
    • delete left-over VMs manually in case you forgot

Creating your own vagrant box

If you - for whatever reason - do not want to use my prebuilt box, you can create your own box very easy!

BEWARE this is for advanced users only!

  • Get packer
  • git checkout the "packer" branch of this repository, follow the README

Author

Shirshendu Mukherjee

Original Authors

Christopher Blum

Daniel Messer - dmesser@redhat.com - Technical Marketing Manager @ Red Hat