/spring-boot-vault-demo

Demo project to show the integration of spring-boot and Hashicorp Vault

Primary LanguageJava

Spring Boot vault demo

Getting started

Create an empty bootstrap.yml:

touch bootstrap.yml

Run gradle to build all necessary files:

./gradlew build

Setup vault

Start the vault container:

docker-compose up --build vault

Initialize Vault:

docker-compose --no-ansi exec -e VAULT_CLI_NO_COLOR=1 -e VAULT_CACERT=/vault/config/ssl/vault.crt.pem vault \
  vault operator init -key-shares=1 -key-threshold=1 > vault_data.txt

Keep the information you will need the root token and unseal key later.

Unseal Vault:

docker-compose exec -e VAULT_CACERT=/vault/config/ssl/vault.crt.pem vault \
  vault operator unseal <unseal key from vault_data.txt>

Check the root token to see if Vault is working:

docker-compose exec -e VAULT_CACERT=/vault/config/ssl/vault.crt.pem vault \
  -e VAULT_TOKEN=<token from vault_data.txt> \
  vault token lookup

You might want to put the docker-compose command line into a shell script, as it will be used a lot later. The following examples assume a ./vault_root.sh with this content:

#!/bin/sh
set -e

docker-compose exec \
  -e VAULT_CACERT=/vault/config/ssl/vault.crt.pem \  
  -e VAULT_TOKEN=<token from vault_data.txt> \
  vault vault "$@"

Run the spring-boot application

docker-compose up --build web

To run the spring-boot application from within an IDE you need to copy build/resources/main/vault-truststore.jks to src/main/resources so that it is on the classpath.

On the first attempt running the application will fail with:

java.lang.IllegalArgumentException: Token (spring.cloud.vault.token) must not be empty

Create a new token for your application

Use vault to create a new application token:

./vault_root.sh token create -policy=default

Put the following block into bootstrap.yml:

spring.cloud.vault:
  token: <token-from-the-vault-output>

To run the spring-boot application from an IDE you will need to add another line:

  host: localhost

take care of the indentation (2 spaces)

Now try again:

./gradlew build && docker-compose up --build web

Now the application starts, but we get the following log messages (lines wrapped for readability):

[RequestedSecret [path='secret/hello-vault', mode=ROTATE]]
  Lease [leaseId='null', leaseDuration=PT0S, renewable=false]
  Status 403 secret/hello-vault: permission denied
[RequestedSecret [path='secret/application', mode=ROTATE]]
  Lease [leaseId='null', leaseDuration=PT0S, renewable=false] 
  Status 403 secret/application: permission denied

This is caused by Vaults default policy ACLs that do not allow access to the secret path.

Add a new policy to allow access for the application, use Vault's API to do this (docker-compose cannot access local files that are not shared with the vault container in advance):

curl -H "X-Vault-Token: <root token from vault_data.txt>" \
  --cacert vault/config/ssl/vault.crt.pem \
  --request PUT --data @./vault/hello-application.json \
  https://localhost:8200/v1/sys/policy/hello-application 

Create a new token for the application:

./vault_root.sh token create -policy=hello-application

Put the new token in bootstrap.yml and restart the application:

docker-compose stop web
./gradlew build
docker-compose up --build web

Now the application starts without errors, check to see the response:

http :8080

HTTP/1.1 200 
Content-Length: 12
Content-Type: text/plain;charset=UTF-8
Date: Mon, 26 Mar 2018 09:21:56 GMT

Hello World!

Adding a secret to vault

The application looks for a value named who at secret/hello-vault, add it to Vault:

./vault_root.sh write secret/hello-vault who=You

Restart the application and see the effect:

http :8080

HTTP/1.1 200 
Content-Length: 10
Content-Type: text/plain;charset=UTF-8
Date: Mon, 26 Mar 2018 09:24:39 GMT

Hello You!

Approle authentication

Vault Approle authentication credentials consists of an approle id and a secret id. Using these credentials the application retrieves a temporary token from Vault. The application role determines which policies are applied for the application.

Setup application role

  • enable approle authentication in vault
    ./vault_root.sh auth enable approle
    
  • create the application role with the existing policy
    ./vault_root.sh write auth/approle/role/hello-vault policies=hello-application
    
  • get the role-id from vault and put it into bootstrap.yml
    ./vault_root.sh read -field=role_id auth/approle/role/hello-vault/role-id
    echo "spring.cloud.vault.app-role.role-id: <role-id>" > bootstrap.yml
    
  • create a secret id and export it as environment variable
    ./vault_root.sh write -f auth/approle/role/hello-vault/secret-id
    export SPRING_CLOUD_VAULT_APP_ROLE_SECRET_ID=<secret-id-from-vault>
    
  • rebuild and run the web application container
    ./gradlew build && docker-compose up --build web
    

Using the PKI secret engine

Vault provides an integrated PKI system. The application uses this secret engine to obtain a dynamically generated server certificate.

Setup vault PKI

  • enable the pki secrets engine
    ./vault_root.sh secrets enable pki
    
  • update the application policy to allow access to relevant pki paths
    curl -H "X-Vault-Token: <root token from vault_data.txt>" \
      --cacert vault/config/ssl/vault.crt.pem \
      --request PUT --data @./vault/hello-application.json \
      https://localhost:8200/v1/sys/policy/hello-application 
    
  • generate a root CA certificate for the pki secrets engine
    ./vault_root.sh write pki/root/generate/internal "common_name=Demo Vault CA"
    
  • setup a pki role for server certificates
    ./vault_root.sh write pki/roles/server client_flag=false ttl=24h
    
  • get the vault pki CA certificate
    curl --cacert vault/config/ssl/vault.crt.pem https://localhost:8200/v1/pki/ca/pem > vaultca.pem
    

Run the spring-boot application

  • build and run
    ./gradlew build && docker-compose up --build web
    
  • test the application
    http --default-scheme https --verify vaultca.pem :8443
    
    HTTP/1.1 200 
    Content-Length: 16
    Content-Type: text/plain;charset=UTF-8
    Date: Mon, 26 Mar 2018 14:04:53 GMT
    
    Hello DevDay 18!
    
  • check that it is really TLS encrypted
    openssl s_client -connect localhost:8443 -CAfile vaultca.pem
    CONNECTED(00000003)
    depth=1 CN = Demo Vault CA
    verify return:1
    depth=0 CN = localhost
    verify return:1
    ---
    Certificate chain
     0 s:/CN=localhost
       i:/CN=Demo Vault CA
     1 s:/CN=Demo Vault CA
       i:/CN=Demo Vault CA
    ---
    ...
    SSL-Session:
        Protocol  : TLSv1.2
        Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    

Using the database secrets engine with PostgreSQL

  • Start the db and restart the vault container:
    docker-compose up db
    docker-compose restart vault 
    
  • Vault will start sealed, unseal it
    docker-compose exec -e VAULT_CLI_NO_COLOR=1 -e VAULT_CACERT=/vault/config/ssl/vault.crt.pem vault \
      vault operator unseal <unseal key from vault_data.txt>
    
  • Setup database and roles
    psql -h localhost -U postgres -d postgres -f pgsql/database_roles.sql --port 15432
    
  • Enable the database secrets engine
    ./vault_root.sh secrets enable database
    
  • Setup the connection from Vault to PostgreSQL
    ./vault_root.sh write database/config/demodb plugin_name=postgresql-database-plugin allowed_roles=springdemo \
      connection_url="postgresql://{{username}}:{{password}}@db:5432/demoapp?sslmode=disable" \
      username="vaultadmin" \
      password="superinsecure"
    

    You should enable SSL for your PostgreSQL server in production and set a better password for the vault user!

  • Create a database role definition for the spring demo application
    ./vault_root.sh write database/roles/springdemo db_name=demodb \
      "creation_statements=CREATE ROLE \"{{name}}\" IN ROLE grp_demo_app_user LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}';" \
      default_ttl=24h max_ttl=72h
    
  • Try to retrieve database credentials
    ./vault_root.sh read database/creds/springdemo
    
  • Try to connect to PostgreSQL with these credentials
    psql -h localhost -U <username-from-vault-output> --port 15432 demoapp
    
  • Update the ACL for the hello-application Vault policy
    curl -H "X-Vault-Token: <root token from vault_data.txt>" \
      --cacert vault/config/ssl/vault.crt.pem \
      --request PUT --data @./vault/hello-application.json \
      https://localhost:8200/v1/sys/policy/hello-application 
    
  • Run the application
    ./gradlew build && docker compose up --build web
    
  • Test the /data/messages REST endpoint to interact with the database
    http --verify vaultca.pem --default-scheme https :8443/data/messages message="Test message"
    http --verify vaultca.pem --default-scheme https :8443/data/messages