GCS - Custom Dual-Region deployment is no functioning thru Terraform
vnachimu opened this issue · 3 comments
TL;DR
Terraform provider is NOT supporting to create a custom dual-region GCS bucket using location field. It works just fine with Google's predefined dual-regions - NAM4, ASIA1, EUR4. When you create a custom-dual region with a location field as US-CENTRAL1+US-EAST1, the terraform state stores the location as US. When we make an update to the bucket next time, there is a mismatch between code and state file, which forces the deployment to be recreated like force replacement.
However, both console and gcloud work fine without any issues. Google handles dual-regions something like this in gcloud-gsutil.
gsutil mb -l MULTI-REGION --placement REGION_1,REGION_2 gs://BUCKET_NAME/
There is an option / switch called "placement" where you can specify the dual-region info. Once the bucket has created, the backend data has been stored as below.
gs://my-bucket/ :
Storage class: STANDARD
Location type: dual-region
Location constraint: US
Placement locations: [US-CENTRAL1
, US-WEST1
]
GCS Terraform is not offering this switch "placement" to specify 2 regions to create a bucket. Therefore, we pass dual-region info like this,
location = US-CENTRAL1+US-EAST1
Expected behavior
When we create a bucket, there should be an option to say the placement locations info in terraform resource. Something like below.
resource "google_storage_bucket" "auto-expire" {
name = "auto-expiring-bucket"
location = "US"
placement = [US-CENTRAL1
, US-WEST1
]
}
Observed behavior
In the current GCS resource of Terraform, we pass dual-region info like below.
resource "google_storage_bucket" "auto-expire" {
name = "auto-expiring-bucket"
location = "US-CENTRAL1+US-EAST1"
}
Terraform Configuration
resource "google_storage_bucket" "auto-expire" {
name = "auto-expiring-bucket"
location = "US-CENTRAL1+US-EAST1"
force_destroy = true
}
Terraform Version
hashicorp/google v4.31.0
Additional information
None
We are experience the same issue. Trying to create a dual region like "US-CENTRAL1+US-EAST1" is stored as US in the state which is causing a destroy/replace on every run. The Buckets are created correctly but the state is incorrect. What is interesting is it was working recently but not any longer. I can look at state files in terraform cloud from a couple weeks ago and it is correct. Possibly down stream dependency bug ?
@vnachimu @tcamechis-cni this is likely something the provider needs to address. Please open an issue in https://github.com/hashicorp/terraform-provider-google
Some details when I reproed this -
resource "google_storage_bucket" "auto-expire" {
name = "auto-expiring-bucket"
location = "US-CENTRAL1+US-EAST1"
force_destroy = true
}
creates the bucket but the API seems to transform given string
{
"kind": "storage#bucket",
"selfLink": "https://www.googleapis.com/storage/v1/b/MY_BUCKET",
"id": "MY_BUCKET",
"name": "MY_BUCKET",
"projectNumber": "0000",
"metageneration": "1",
"location": "NAM4",
"storageClass": "STANDARD",
"etag": "CAE=",
"timeCreated": "2022-08-09T20:42:53.164Z",
"updated": "2022-08-09T20:42:53.164Z",
"iamConfiguration": {
"bucketPolicyOnly": {
"enabled": false
},
"uniformBucketLevelAccess": {
"enabled": false
},
"publicAccessPrevention": "inherited"
},
"locationType": "dual-region",
"customPlacementConfig": {
"dataLocations": [
"US-CENTRAL1",
"US-EAST1"
]
},
"rpo": "DEFAULT"
}
Initial input of US-CENTRAL1+US-EAST1
seems to be mapped to customPlacementConfig.dataLocations
You may also be able to workaround this in your config by manually mapping
locals {
dual_location_map = {
"US-CENTRAL1+US-EAST1" : "NAM4"
}
}
resource "google_storage_bucket" "auto-expire" {
name = "auto-expiring-bucket"
location = local.dual_location_map["US-CENTRAL1+US-EAST1"]
force_destroy = true
}
or using ignore changes https://www.terraform.io/language/meta-arguments/lifecycle#ignore_changes