googleapis/python-dataproc

DataProc: Cluster config doesn't show lifecycle config for clusters created outside 'global' region

jonescameron opened this issue · 2 comments

Trying to write a function in Python that parses cluster config files, I've noticed that when I import dataproc_v1beta2, I can see lifecycle info in the cluster config with the format:

lifecycle_config {
  idle_delete_ttl {
    seconds: 21600
  }
}

which is perfect, however when getting config from clusters created in any region other than 'global', this information is not present.

Code to reproduce:

from google.cloud import dataproc_v1beta2
from google.cloud.dataproc_v1.gapic.transports import cluster_controller_grpc_transport as ccgt

project = 'myProject'
region = 'europe-west1'
transport = ccgt.ClusterControllerGrpcTransport(address='{}-dataproc.googleapis.com:443'.format(region))

client = dataproc_v1beta2.ClusterControllerClient(transport)
c1 = client.get_cluster(project, region, 'test-cluster-in-europe-sched')
c2 = client.get_cluster(project, region, 'test-cluster-in-europe-no-sched')

region = 'global'

client = dataproc_v1beta2.ClusterControllerClient()
c3 = client.get_cluster(project, region, 'test-cluster-with-schedule')
c4 = client.get_cluster(project, region, 'test-cluster-without-schedule')

c1cfg = c1.config
c2cfg = c2.config
c3cfg = c3.config
c4cfg = c4.config

When running print() on these 4 ClusterConfigs, lifecycle_config only appears on c3, where I would have expected it on c1 also.

Thanks for your time.

Thanks for the report!

As of 0.7.0 this should be fixed for dataproc_v1beta2.