Growing Metrics Volume Cost

[michal-billtech]

TL;DR

Usage of the module incures a constantly growing cost of Metric Volume SKU.

Expected behavior

I'd expect the costs to stay at constant level unless the cluster is scaled up or down.

Observed behavior

As seen on the attached screenshot, there is a constantly growing cost of Metric Volume SKU. There was no growth in Vault usage during this period. On November 24th I took away the Monitoring Metric Writer role from Vault SA and that is when the Metric Volume usage dropped to 0.

Is it expected for the Metric Volume usage to grow constantly like this? It seems strange to me that the metrics would cost more than the VM Vault is running on. What are those metrics needed for? What harm could it do if I disabled them on production environment?

Terraform Configuration

module "vault" {
  source     = "terraform-google-modules/vault/google"
  version    = "~>6.0.0"
  project_id = var.project_id
  region     = var.region

  user_startup_script = "sudo apt-get update && sudo apt-get install jq -y"

  load_balancing_scheme = "INTERNAL"
  vault_version         = "1.8.5"

  network                   = module.common-vpc.network_self_link
  network_subnet_cidr_range = var.vault-subnetwork-range

  kms_keyring    = ""
  kms_crypto_key = ""

  domain = ""

  allow_public_egress = true

  ssh_allowed_cidrs = [
    ""
  ]

  storage_bucket_class    = "REGIONAL"
  storage_bucket_location = var.region

  vault_allowed_cidrs = [""]

  tls_ca_subject      = {
    "common_name" : "",
    "country" : "",
    "locality" : "",
    "organization" : "",
    "organizational_unit" : "",
    "postal_code" : "",
    "province" : "",
    "street_address" : [
      ""
    ]
  }
  tls_cn              = ""
  tls_dns_names       = [
    ""
  ]
  tls_save_ca_to_disk = false

}

module "common-vpc" {
  source     = "terraform-google-modules/network/google"
  version    = "3.2.0"
  project_id = var.project_id

  network_name            = ""
  auto_create_subnetworks = false

  subnets = [
    {
      subnet_name           = ""
      subnet_ip             = ""
      subnet_region         = ""
      subnet_private_access = "true"
    }
  ]
}

### Terraform Version

```sh
Terraform v1.0.11
on linux_amd64
+ provider registry.terraform.io/hashicorp/google v3.90.1
+ provider registry.terraform.io/hashicorp/google-beta v4.1.0
+ provider registry.terraform.io/hashicorp/local v2.1.0
+ provider registry.terraform.io/hashicorp/null v3.1.0
+ provider registry.terraform.io/hashicorp/random v3.1.0
+ provider registry.terraform.io/hashicorp/template v2.2.0
+ provider registry.terraform.io/hashicorp/tls v2.1.1

Additional information

Using the metrics explorer I've been able to track down the growing metric to be custom.googleapis.com/statsd/gauge

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days