Vault root_token.env is missing

[RyanSiu1995]

Hi, I have followed all the guides you have written for us.
First of all, thank you for your guides. I found it helpful for me.
After the deployment of vault in GKE, I could not find the root_token.env on my GCS.
The only way I could do to grep the token was first to initialize the vault with dev mode. The token would appear on the log.
Any idea why the root_token.env is missing by following the guides?

[sethvargo]

Do you mean root-token.enc? It's enc, not env

[RyanSiu1995]

Yes, I mean root-token.enc. I cannot find it after the initialization.

[sethvargo]

Did the Terraform run complete successfully? What do the Vault logs show?

[RyanSiu1995]

The log for the vault is here

=> Vault server configuration:

      GCP KMS Crypto Key: vault-init
        GCP KMS Key Ring: vault
         GCP KMS Project: sre-001
          GCP KMS Region: us-east1
               Seal Type: gcpckms
             Api Address: https://XXXXXXX
                     Cgo: disabled
         Cluster Address: https://10.0.1.18:8201
              Listener 1: tcp (addr: "127.0.0.1:8200", cluster address: "127.0.0.1:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
              Listener 2: tcp (addr: "10.0.1.18:8200", cluster address: "10.0.1.18:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "enabled")
               Log Level: debug
                   Mlock: supported: true, enabled: true
                 Storage: gcs (HA available)
                 Version: Vault v1.0.3
             Version Sha: 85909e3373aa743c34a6a0ab59131f61fd9e8e43

==> Vault server started! Log data will stream in below:

2019-07-09T08:23:28.495Z [DEBUG] storage.gcs: configuring backend
2019-07-09T08:23:28.495Z [DEBUG] storage.gcs: configuration: bucket=sre-001-vault-storage chunk_size=8388608 ha_enabled=true max_parallel=0
2019-07-09T08:23:28.495Z [DEBUG] storage.gcs: creating client
2019-07-09T08:23:28.692Z [DEBUG] storage.cache: creating LRU cache: size=0
2019-07-09T08:23:28.776Z [DEBUG] cluster listener addresses synthesized: cluster_addresses=[127.0.0.1:8201, 10.0.1.18:8201]
2019-07-09T08:23:28.920Z [INFO]  core: stored unseal keys supported, attempting fetch
2019-07-09T08:23:29.158Z [INFO]  core: vault is unsealed
2019-07-09T08:23:29.158Z [INFO]  core: entering standby mode
2019-07-09T08:23:29.190Z [INFO]  core: unsealed with stored keys: stored_keys_used=1
2019-07-09T08:23:34.483Z [INFO]  core: acquired lock, enabling active operation
2019-07-09T08:23:34.754Z [DEBUG] core: generating cluster private key
2019-07-09T08:23:34.773Z [DEBUG] core: generating local cluster certificate
2019-07-09T08:23:34.981Z [INFO]  core: post-unseal setup starting
2019-07-09T08:23:35.010Z [DEBUG] core: clearing forwarding clients
2019-07-09T08:23:35.010Z [DEBUG] core: done clearing forwarding clients
2019-07-09T08:23:35.046Z [INFO]  core: loaded wrapping token key
2019-07-09T08:23:35.046Z [INFO]  core: successfully setup plugin catalog: plugin-directory=
2019-07-09T08:23:35.170Z [INFO]  core: successfully mounted backend: type=kv path=secret/
2019-07-09T08:23:35.170Z [INFO]  core: successfully mounted backend: type=system path=sys/
2019-07-09T08:23:35.170Z [INFO]  core: successfully mounted backend: type=identity path=identity/
2019-07-09T08:23:35.170Z [INFO]  core: successfully mounted backend: type=kv path=kv/
2019-07-09T08:23:35.170Z [INFO]  core: successfully mounted backend: type=cubbyhole path=cubbyhole/
2019-07-09T08:23:35.576Z [INFO]  core: successfully enabled credential backend: type=token path=token/
2019-07-09T08:23:35.576Z [INFO]  core: successfully enabled credential backend: type=kubernetes path=kubernetes/
2019-07-09T08:23:35.576Z [INFO]  core: restoring leases
2019-07-09T08:23:35.577Z [DEBUG] expiration: collecting leases
2019-07-09T08:23:35.577Z [INFO]  rollback: starting rollback manager
2019-07-09T08:23:35.664Z [DEBUG] identity: loading entities
2019-07-09T08:23:35.723Z [DEBUG] identity: entities collected: num_existing=0
2019-07-09T08:23:35.724Z [INFO]  identity: entities restored
2019-07-09T08:23:35.724Z [DEBUG] identity: identity loading groups
2019-07-09T08:23:35.795Z [DEBUG] identity: groups collected: num_existing=0
2019-07-09T08:23:35.795Z [INFO]  identity: groups restored
2019-07-09T08:23:35.825Z [DEBUG] core: starting cluster listeners
2019-07-09T08:23:35.826Z [DEBUG] core: cluster listener setup function
2019-07-09T08:23:35.826Z [DEBUG] core: clearing forwarding clients
2019-07-09T08:23:35.826Z [DEBUG] core: done clearing forwarding clients
2019-07-09T08:23:35.826Z [DEBUG] core: leaving cluster listener setup function
2019-07-09T08:23:35.826Z [INFO]  core: post-unseal setup complete
2019-07-09T08:23:35.826Z [INFO]  core: starting listener: listener_address=127.0.0.1:8201
2019-07-09T08:23:35.827Z [INFO]  core: serving cluster requests: cluster_listen_address=127.0.0.1:8201
2019-07-09T08:23:35.827Z [INFO]  core: starting listener: listener_address=10.0.1.18:8201
2019-07-09T08:23:35.827Z [INFO]  core: serving cluster requests: cluster_listen_address=10.0.1.18:8201

I did not run the terraform but follow the guide you have written. The vault did launch and I can get the token from the hack I have done. It worked pretty well but the only issue was the root-token.enc

[sethvargo]

Can you do a gsutil ls on the bucket and show which values are there?

[RyanSiu1995]

Here you are.

$ gsutil ls gs://sre-001-vault-storage/
gs://sre-001-vault-storage/auth/
gs://sre-001-vault-storage/core/
gs://sre-001-vault-storage/logical/
gs://sre-001-vault-storage/sys/

[sethvargo]

I'm really not sure. I just ran through the tutorial myself and can't reproduce the issue. I'm not sure how to help, sorry! 😦

[RyanSiu1995]

It's fine. Let me have a retry on it first. Thanks a lot!