GCS - Custom Dual-Region deployment is no functioning thru Terraform

[vnachimu]

TL;DR

Terraform provider is NOT supporting to create a custom dual-region GCS bucket using location field. It works just fine with Google's predefined dual-regions - NAM4, ASIA1, EUR4. When you create a custom-dual region with a location field as US-CENTRAL1+US-EAST1, the terraform state stores the location as US. When we make an update to the bucket next time, there is a mismatch between code and state file, which forces the deployment to be recreated like force replacement.

However, both console and gcloud work fine without any issues. Google handles dual-regions something like this in gcloud-gsutil.

gsutil mb -l MULTI-REGION --placement REGION_1,REGION_2 gs://BUCKET_NAME/

There is an option / switch called "placement" where you can specify the dual-region info. Once the bucket has created, the backend data has been stored as below.

gs://my-bucket/ :
Storage class: STANDARD
Location type: dual-region
Location constraint: US
Placement locations: [US-CENTRAL1, US-WEST1]

GCS Terraform is not offering this switch "placement" to specify 2 regions to create a bucket. Therefore, we pass dual-region info like this,

location = US-CENTRAL1+US-EAST1

Expected behavior

When we create a bucket, there should be an option to say the placement locations info in terraform resource. Something like below.

resource "google_storage_bucket" "auto-expire" {
name = "auto-expiring-bucket"
location = "US"
placement = [US-CENTRAL1, US-WEST1]
}

Observed behavior

In the current GCS resource of Terraform, we pass dual-region info like below.

resource "google_storage_bucket" "auto-expire" {
name = "auto-expiring-bucket"
location = "US-CENTRAL1+US-EAST1"
}

Terraform Configuration

resource "google_storage_bucket" "auto-expire" {
  name          = "auto-expiring-bucket"
  location      = "US-CENTRAL1+US-EAST1"
  force_destroy = true
}

Terraform Version

hashicorp/google v4.31.0

Additional information

None

[tcamechis-cni]

We are experience the same issue. Trying to create a dual region like "US-CENTRAL1+US-EAST1" is stored as US in the state which is causing a destroy/replace on every run. The Buckets are created correctly but the state is incorrect. What is interesting is it was working recently but not any longer. I can look at state files in terraform cloud from a couple weeks ago and it is correct. Possibly down stream dependency bug ?

[bharathkkb]

@vnachimu @tcamechis-cni this is likely something the provider needs to address. Please open an issue in https://github.com/hashicorp/terraform-provider-google

[bharathkkb]

Some details when I reproed this -

resource "google_storage_bucket" "auto-expire" {
  name          = "auto-expiring-bucket"
  location      = "US-CENTRAL1+US-EAST1"
  force_destroy = true
}

creates the bucket but the API seems to transform given string

{
  "kind": "storage#bucket",
  "selfLink": "https://www.googleapis.com/storage/v1/b/MY_BUCKET",
  "id": "MY_BUCKET",
  "name": "MY_BUCKET",
  "projectNumber": "0000",
  "metageneration": "1",
  "location": "NAM4",
  "storageClass": "STANDARD",
  "etag": "CAE=",
  "timeCreated": "2022-08-09T20:42:53.164Z",
  "updated": "2022-08-09T20:42:53.164Z",
  "iamConfiguration": {
    "bucketPolicyOnly": {
      "enabled": false
    },
    "uniformBucketLevelAccess": {
      "enabled": false
    },
    "publicAccessPrevention": "inherited"
  },
  "locationType": "dual-region",
  "customPlacementConfig": {
    "dataLocations": [
      "US-CENTRAL1",
      "US-EAST1"
    ]
  },
  "rpo": "DEFAULT"
}

Initial input of US-CENTRAL1+US-EAST1 seems to be mapped to customPlacementConfig.dataLocations

You may also be able to workaround this in your config by manually mapping

locals {
  dual_location_map = {
  "US-CENTRAL1+US-EAST1" : "NAM4"
  }
}

resource "google_storage_bucket" "auto-expire" {
  name          = "auto-expiring-bucket"
  location      = local.dual_location_map["US-CENTRAL1+US-EAST1"]
  force_destroy = true
}

or using ignore changes https://www.terraform.io/language/meta-arguments/lifecycle#ignore_changes