run_destroy_command not invoking gcloud correctly
shybbko opened this issue · 1 comments
shybbko commented
TL;DR
Executing module.composer_master_authorized_networks.module.gcloud.null_resource.run_destroy_command[0] results in exit status 127. Output: /bin/sh: gcloud: not found in an environment where gcloud is not available out of the box even though the module is supposed to automatically download missing dependency (gcloud).
Expected behavior
No errors should get thrown, gcloud should get downloaded and the command should get executed properly.
Observed behavior
│ Error: local-exec provisioner error
│
│ with module.composer_master_authorized_networks.module.gcloud.null_resource.run_destroy_command[0],
│ on .terraform/modules/composer_master_authorized_networks.gcloud/main.tf line 258, in resource "null_resource" "run_destroy_command":
│ 258: provisioner "local-exec" {
│
│ Error running command 'PATH=/google-cloud-sdk/bin:$PATH
│ gcloud container clusters update --project=projectname
│ --zone=us-central1-f
│ projects/projectname/zones/us-central1-f/clusters/us-central1-projectname-cl-randomid-gke
│ --enable-master-authorized-networks
│ ': exit status 127. Output: /bin/sh: gcloud: not found
Terraform Configuration
Basically just running V1 to V2 upgrade. Not substituting vars as they're not relevant here (also no sensitive values were requested).
V1:
module "composer_environment" {
source = "terraform-google-modules/composer/google//modules/create_environment_v1"
version = "~> 3.4.0"
project_id = var.project_id
composer_env_name = var.composer_env_name
region = var.region
composer_service_account = google_service_account.composer_service_account.email
network_project_id = var.network_project_id
network = var.vpc_network
subnetwork_region = var.region
subnetwork = var.composer_subnetwork
zone = var.composer_zone
use_ip_aliases = true
enable_private_endpoint = true
pod_ip_allocation_range_name = "pods"
service_ip_allocation_range_name = "services"
image_version = "composer-1.20.11-airflow-2.4.3"
web_server_ipv4_cidr = var.composer_webserver_sb
cloud_sql_ipv4_cidr = var.composer_cloud_sql_sb
master_ipv4_cidr = var.composer_master_sb
disk_size = "50"
machine_type = "n1-standard-8"
node_count = "4"
tags = ["any-out", "ssh-in"]
depends_on = [
google_service_account.composer_service_account,
google_project_service_identity.composer_service_identity
]
pypi_packages = {
pdpyras = "",
google-api-python-client = "",
oauth2client = "",
google-cloud = "",
google-cloud-secret-manager = "",
google-cloud-storage = "",
google-cloud-dataform = "",
apache-airflow-providers-google = ""
}
airflow_config_overrides = {
api-enable_experimental_api = "True",
api-auth_backend = "airflow.api.auth.backend.default"
}
}
V2 (this throws errors, however reverting back to V1 would also throw an error):
module "composer_environment" {
source = "terraform-google-modules/composer/google//modules/create_environment_v2"
version = "~> 3.4.0"
project_id = var.project_id
composer_env_name = var.composer_env_name
region = var.region
composer_service_account = google_service_account.composer_service_account.email
network_project_id = var.network_project_id
network = var.vpc_network
subnetwork_region = var.region
subnetwork = var.composer_subnetwork
enable_private_endpoint = true
use_private_environment = true
pod_ip_allocation_range_name = "pods"
service_ip_allocation_range_name = "services"
image_version = "composer-2.2.1-airflow-2.4.3"
cloud_sql_ipv4_cidr = var.composer_cloud_sql_sb
master_ipv4_cidr = var.composer_master_sb
tags = ["any-out", "ssh-in"]
depends_on = [
google_service_account.composer_service_account,
google_project_service_identity.composer_service_identity
]
pypi_packages = {
pdpyras = "",
google-api-python-client = "",
oauth2client = "",
google-cloud = "",
google-cloud-secret-manager = "",
google-cloud-storage = "",
google-cloud-dataform = "",
apache-airflow-providers-google = ""
}
airflow_config_overrides = {
api-enable_experimental_api = "True",
api-auth_backend = "airflow.api.auth.backend.default"
}
}
Terraform Version
Terraform v1.0.10
on linux_amd64
Additional information
This is the relevant plan output:
# module.composer_master_authorized_networks.module.gcloud.null_resource.run_destroy_command[0] must be replaced
-/+ resource "null_resource" "run_destroy_command" {
~ id = "someotherid" -> (known after apply)
~ triggers = { # forces replacement
~ "destroy_cmd_body" = "container clusters update --project=projectname --zone=us-central1-f projects/projectname/zones/us-central1-f/clusters/us-central1-projectname-cl-randomid-gke --enable-master-authorized-networks" -> (known after apply)
# (2 unchanged elements hidden)
}
}
Thow whole plan also consists of
# module.composer_environment.google_composer_environment.composer_env must be replaced
-/+ resource "google_composer_environment" "composer_env" {
~ id = "projects/projectname/locations/us-central1/environments/projectname-cl01" -> (known after apply)
name = "projectname-cl01"
# (3 unchanged attributes hidden)
~ config {
(...)
}
# module.composer_environment.google_project_iam_member.composer_agent_service_account[0] will be created
+ resource "google_project_iam_member" "composer_agent_service_account" {
(...)
}
# module.composer_master_authorized_networks.module.gcloud.null_resource.run_command[0] must be replaced
-/+ resource "null_resource" "run_command" {
(...)
}
}
however if the destroy_command is executed manually from an environment with gcloud available, the other bits get deployed fine as well, without complaining about gcloud not being there.
github-actions commented
This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days