This repository is the definition of OpenMRS community infrastructure (infra-as-code). We are using terraform to generate network infra, Openstack VMs (Jetstream IU and TACC), DNS and backup resources.
You can also check https://github.com/bmamlin/jetstream-api-cli.git to check the results in Openstack.
This repository can generate documentation for all VMs created, and it's deployed to https://docs.openmrs.org/infrastructure/vms.html
Check Provision new machine and Guidelines for New Servers for more details on how to create machines (these docs are not public).
Before you can use this repository, you need:
- TACC (Jetstream) credentials - check internal wiki for Jetstream access details
- Be included in git crypt in this repository (to access secrets)
You need to have installed:
- git-crypt
- ruby (2.0+)
- thor (
gem install thor)
To install terraform and initial setup (needed only once)
$ ./build.rb install
# add your TACC credentials to conf/openrc-personal
$ vi conf/openrc-personal
# verify file is not encrypted
$ cat conf/openrc
# to init terraform in all VMs (takes time)
$ ./build.rb init_all
# to init terraform in a single VM (e.g. narok)
$ ./build.rb init narok
To undo the changes from the previous commands:
$ ./build.rb clean
To create a new stack test:
$ ./build.rb create test
To run terraform plan (and see what changed on your stack) on a base-network stack:
$ ./build.rb plan base-network/
To run terraform apply (and apply changes to a stack) on a base-network stack:
$ ./build.rb apply base-network/
To see all available commands:
$ ./build.rb
Forcing a VM to be reprovisioned:
$ ./build.rb taint-vm <stack>
$ ./build.rb plan <stack>
$ ./build.rb apply <stack>
To SSH a machine before running ansible:
$ ssh -i conf/provisioning/ssh/terraform-api.key root@<machine>
After ansible, you should use your regular user.
To generate and update VM documentation:
$ ./build docs
$ ./build plan docs
$ ./build apply docs
- build.rb: build helper (thor) file
- conf/ : configuration files and authentication files
- conf/template-stack: base files used when creating new stacks
- conf/provisioning: keys and helpers to run ansible when provisioning a machine. Each
- modules/: terraform modules
- modules/single-machine: terraform module to generate a machine, with a A DNS record.
- global-variables.tf: global terraform variables symlinked on each stack
- base-network/ : stack basic infrastructure (network, subnets, routers)
- other-stacks/: each machine should have a directory defined in here. Each folder should be a different state/stack file.
Each stack should be more or less:
- main.tf: terraform resources
- outputs.tf: stack outputs (can be used by other stacks as input params)
- variables.tf: terraform variables
- global-variables.tf: symlink to the top level global file.
- Verify that the entry doesn't already exist in our DNS provider.
Ansible configures and secures our SSH configuration, so root cannot SSH anymore.
Change ./global-variables and use your username and (passphrase-less) key. Do not commit this change.
- DNS CNAME records cannot be imported by terraform, so they have to be deleted in our DNS server before using them in a stack.
- Updating a DNS resource doesn't work in terraform, it appears to be a bug. You need to either delete and create, or change manually.
- Don't use the DNS redirect. It doesn't support HTTPS.
- Use /etc/ansible/facts.d/ files to export data to ansible. If the files should be modified, you can do it manually.
Some resources are necessary to run terraform, so they were created manually:
- S3 bucket to keep terraform state (versioned)
- User to interact with bucket (access via bucket policy)
- DNS domains (not defined in terraform provider)