/ansible-playbook-vault

Playbooks to setup a (development) vault environment.

Primary LanguageHCL

Vault Upgrade playbook

Try upgrades of Vault over DR and HA clusters.

Overview

This setup consists of:

  • 2 "global" loadbalancers
  • 2*2 "local" loadbalancers
  • 2 Highly Available (HA) clusters
  • 1 Disaster Recovery (DR) cluster
  • 2*1 HashiCorp Vault leaders, 1 in each HA cluster.
  • 2*4 HashiCorp Vault followers, 4 in each HA cluster.

The loadbalancers are setup to:

  1. Allow a loadbalancer outage.
  2. Allow a datacenter outage.
  +--- loadbalancer-0 ---+     +--- loadbalancer-1 ---+
  |                      +-+ +-+                      |
  +----------------------+ | | +----------------------+
+--- loadbalancer-a-0 ---+ | | +--- loadbalancer-b-0 ---+
|                        +-+ +-+                        |
+------------------------+ | | +------------------------+
+--- loadbalancer-a-1 ---+ | | +--- loadbalancer-b-1 ---+
|                        +-+ +-+                        |
+------------------------+     +------------------------+

The Vault cluster is setup like displayed below.

+------------------- DR cluster --------------------+
| +--- HA cluster "A" ---+ +--- HA cluster "B" ---+ |
| | +--- vault-a-0 ---+  | | +--- vault-b-0 ---+  | |
| | | leader          |  | | | leader          |  | |
| | +-----------------+  | | +-----------------+  | |
| | +--- vault-a-1 ---+  | | +--- vault-b-1 ---+  | |
| | | follower        |  | | | follower        |  | |
| | +-----------------+  | | +-----------------+  | |
| | +--- vault-a-2 ---+  | | +--- vault-b-3 ---+  | |
| | | follower        |  | | | follower        |  | |
| | +-----------------+  | | +-----------------+  | |
| | +--- vault-a-3 ---+  | | +--- vault-b-3 ---+  | |
| | | follower        |  | | | follower        |  | |
| | +-----------------+  | | +-----------------+  | |
| | +--- vault-a-4 ---+  | | +--- vault-b-4 ---+  | |
| | | follower        |  | | | follower        |  | |
| | +-----------------+  | | +-----------------+  | |
| +----------------------+ +----------------------+ |
+---------------------------------------------------+

Prepare

Download or update the Ansible roles.

ansible-galaxy install -r roles/requirements.yml -f

The state of the used roles:

Role name GitHub Action GitLab CI Version
bootstrap github gitlab version
common github gitlab version
core_dependencies github gitlab version
digitalocean-agent github gitlab version
environment github gitlab version
haproxy github gitlab version
hashicorp github gitlab version
keepalived github gitlab version
logwatch github gitlab version
service github gitlab version
users github gitlab version
vault github gitlab version

Test

Run the playbook against the virtual machines.

# Setup the machines with Vault 1.4.2.
# 1.4.2 is an old version, so you can try upgrades.
#
# First setup two HA clusters
./1_install_vault.yml
# This playbook saves the unseal_keys in `group_vars/vault_(a|b)/vault.yml`.
# Make a backup, save the results to /root/.
./2_backup_vault.yml
# Upgrade the vault clusters.
./3_upgrade_vault.yml
# Setup DR cluster. Here is where cluster "a" and "b" are related.
./4_setup_dr.yml
# Generate a failover token, required to do a DR failover.
./5_generate_failover_token.yml

HashiCorp Vault enterprise seals itself after 30 minutes of use without entering a license. To develop procedures, restart and unseal Vault 30 minutes after starting up.

./9_restart_and_unseal.yml

Cleanup

Throw away the machines.

cd terraform
terraform destroy