[GCP] Migrate machine type to e2 family to save costs
Bobgy opened this issue · 8 comments
Bobgy commented
Got errors:
*kf-ci-management
--------------------
<root> https:/github.com/kubeflow/testing.git/acm-repos/kf-ci-management@master
ERROR 472c24c0
Error: KNV2010: unable to update resource: could not patch: admission webhook "deny-immutable-field-updates.cnrm.cloud.google.com" denied the request: cannot make changes on immutable fields: [initialNodeCount nodeConfig]
source: namespaces/issue-label-bot-dev/container.cnrm.cloud.google.com_v1beta1_containercluster_code-intelligence.yaml
namespace: issue-label-bot-dev
metadata.name: code-intelligence
group: container.cnrm.cloud.google.com
version: v1beta1
kind: ContainerCluster
So I cannot change nodeInitial or nodeConfig fields, I'll add them back.
ContainerNodepool change fails with error
status:
conditions:
- lastTransitionTime: "2021-01-19T05:33:07Z"
message: reference IAMServiceAccount /code-intelligence-vm@issue-label-bot-dev.iam.gserviceaccount.com
is not found
reason: DependencyNotFound
status: "False"
type: Ready
Looks like the service account format is incorrect
Bobgy commented
Done, actions taken for code-intelligence cluster:
- update container node pool to e2-standard-8 machine type, change nodepool name, because it's an immutable field
- delete default nodepool
- disable node autoprovisioning, delete auto-provisioned nodepool
Bobgy commented
I tried to disable node autoprovisioning via config connector, but it keeps failing with GoogleCloudPlatform/k8s-config-connector#277.
I had to disable auto provisioning manually for now.
Bobgy commented
TODO: delete auto provisioned nodepool from issue-label-bot cluster.
Bobgy commented
For record, 3h10min spent on e2 migration.