This repo contains a dockerfile for building a docker image, and an job spec to run it under nomad to create and renew letsencrypt certificates, using vault for backend storage
- Nomad jobs can inject the certificates using the Nomad-Vault integration without the applications needing to be aware of letsencrypt
- Use of DNS RFC2136 means the applications themselves don't have to be publicly accessible
- Centralising the registration and renewal of certificates means dns keys for dynamic updates need only be configured in one place
This is cribbed heavily from [[https://developer.epages.com/blog/tech-stories/managing-lets-encrypt-certificates-in-vault/]] and tweaked to use rfc2136 instead of dnsimple and to run under nomad.
Start off by cloning this repository
Add the CA certificate used by your vault server to the root of your git clone as ca.crt
.
# certbot-nomad.policy
path "secret/metadata/lets-encrypt/*" {
capabilities = ["create", "read", "update", "delete", "list"]
}
path "secret/data/lets-encrypt/*" {
capabilities = ["create", "read", "update", "delete", "list"]
}
Configure the policy in vault with:
vault policy write certbot-nomad certbot-nomad.policy
Generate a tsig key to authenticate the dynamic update with
tsig-keygen -a hmac-sha512 certbot-nomad
This will print output like the following:
key "certbot-nomad" {
algorithm hmac-sha512;
secret "V54YyKknsuHiSDFiIC2E4uzWHjxxusN2jJRcbYF6MYVkjGpejW8D0ECP09VI7hiwYQbhwZ0aA9nmXlekzuKRLA==";
};
Keep a copy of this output which you'll need for configuring the nameserver. Also, take the secret and write to vault, along with the key name (certbot-nomad
as used in the tsig-keygen
command), and server name that dynamic updates should be sent to
vault kv put secret/lets-encrypt/tsig-key \
name=certbot-nomad \
key=V54YyKknsuHiSDFiIC2E4uzWHjxxusN2jJRcbYF6MYVkjGpejW8D0ECP09VI7hiwYQbhwZ0aA9nmXlekzuKRLA== \
server nameserver.example.com
Allow updates using the tsig key you generated. For bind, this is done by adding the full output from tsig-keygen
above into your config. Then allow dynamic updates on the specific zone with:
allow-update { key certbot; };
Build a docker image and push it your registry with:
docker build -t myregistry.example.com:5000/certbot-nomad:latest .
docker push myregistry.example.com:5000/certbot-nomad
This image is customised with your CA certificate. There's nothing in it that's sensitive, so this can be publihed publicly, but since it contains your CA cert it's only usable with your vault cluster.
A future improvement would be to pass in the CA certificate at runtime.
Run an instance of the container with
docker run --rm -it --name certbot-vault \
-e "VAULT_ADDR=http://dev-vault:8200" \
-e "VAULT_TOKEN=${VAULT_TOKEN}" \
--network certbot-vault-net \
certbot-vault sh
Create a Let's Encrypt account:
certbot register --non-interactive --agree-tos -m webmaster@example.com
Register the state in Vault:
export ACCOUNT_PARENT_PATH=/etc/letsencrypt/accounts/acme-v02.api.letsencrypt.org/directory
export ACCOUNT_ID=$(ls $ACCOUNT_PARENT_PATH)
vault kv put secret/lets-encrypt/account/extra_details "account_id=$ACCOUNT_ID"
for i in meta private_key regr; do
vault kv put "secret/lets-encrypt/account/$i" "@$ACCOUNT_PARENT_PATH/$ACCOUNT_ID/$i.json"
done
Customise the nomad job with:
- Your nomad datacenter name
- Your registry URL for your docker image
- Your Vault URL
Submit the job to nomad with:
nomad job run certbot-nomad.hcl
Run a copy of the container with:
docker run --rm -it --name certbot-vault \
-e "VAULT_ADDR=http://dev-vault:8200" \
-e "VAULT_TOKEN=${VAULT_TOKEN}" \
--network certbot-vault-net \
certbot-vault sh
And request a certificate:
/usr/local/bin/intialize.sh
certbot certonly --dns-rfc2136 --dns-rfc2136-credentials /etc/letsencrypt/creds.ini \
--deploy-hook /etc/letsencrypt/renewal-hooks/deploy/00-update-vault.sh \
-d host.example.com -d alias.example.com