terraform-google-modules/terraform-google-vault

Vault Instance Group keeps transforming

raj-saxena opened this issue · 1 comments

Hi,

We are on the latest version of the module. Things generally work fine but recently, we have started seeing a high number of errors of the type - connection refused.
We also noticed that the vault instances in the instance group are always unhealthy and the health check status is also always Timeout. We suspect that due to this, the instances are constantly recreated and that might be the reason why the connection requests from services time out.

Vault-recreate-issue.mov

To be 100% sure, I verified over a period of days that the creation time of the instances is within the last hour & the Internal IP also keeps changing which does mean that the nodes are being recreated.
Is this the ideal and wanted behaviour?

I suspected that there might be a firewall issue but I get success when I check the status from within the network with curl "https://<vault-ip>:8200/v1/sys/health?uninitcode=200&standbyok=true"

TIA for any solutions, hints & suggestions.

Sorry, this was a misconfiguration on our side. We kept vault_allowed_cidrs to empty list & hence the health checks were unable to reach the instance nodes.
Once we added 35.191.0.0/16 & 130.211.0.0/22 as the source IPs (docs), the instances became healthy & the instance groups became stable.
Screenshot 2021-03-02 at 06 06 16