dataproc api does not set lifecycle value while cluster is creating

[cci-ykula]

I am trying to create cluster by rest api. we want to add lifecycle config as following. The rest request creates cluster but we could not see lifecycle config.

Our request is like :
` cluster = {
"project_id": project,
"cluster_name": cluster_name_new,
"config": {
"master_config": {"num_instances": 1, "machine_type_uri": MACHINE_TYPE_M}, # "n1-standard-2"
"worker_config": {"num_instances": 2, "machine_type_uri": MACHINE_TYPE_W}, #
"gce_cluster_config": {"subnetwork_uri": "projects//regions/europe-west1/subnetworks/*", "zone_uri": "europe-west1-d", "service_account_scopes": [ "https://www.googleapis.com/auth/cloud-platform" ] },
"lifecycle_config": {"idle_delete_ttl": d.FromSeconds(10800)},
"initialization_actions": [ { "executable_file": "gs://dataproc-init-config/init_actions.sh" } ]
},
"labels": {"env": "connected-planning-sparc-cluster"},

}`

After sending request , it seem as following on cloud console. Unfortunately 'Scheduled deletion' value is Off. it should be on .

we use beta version like that : from google.cloud import dataproc_v1beta2 as dataproc
Our application is dockerized. we use python:3.8-slim image.
Our requirement file is like:
backoff==1.10.0 grpcio==1.37.1 google-auth==1.30.2 google-auth-httplib2==0.1.0 google-cloud==0.34.0 google-cloud-storage==1.38.0 google-cloud-dataproc==2.4.0

[meredithslota]

https://cloud.google.com/dataproc/docs/reference/rest/v1/ClusterConfig says that the lifecycle config is lifecycleConfig — I suspect that the lifecycle config is not being properly set (your code above uses "lifecycle_config"). You can check to see if you have scheduled deletion on by using clusters.list as per this documentation page: https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/scheduled-deletion#rest-api_1 — can you let us know if this solves the issue?

I would also recommend using v1 instead of v1beta2 but I don't think that's the problem here, just a general recommendation.

[novikovaTanya]

d.FromSeconds(10800) returns null. If you want to use duration, you would probably want to do something like this:

>>> from google.protobuf.duration_pb2 import Duration
>>> d = Duration()
>>> d.FromSeconds(10800)
>>> d
seconds: 10800

and the you can use it this: "lifecycle_config": {"idle_delete_ttl": d},

[cci-ykula]

hi @novikovaTanya . you are right. It is solved. Thank you for everything.