kubernetes/cloud-provider-aws

Deprecate and remove AWS cloud-controller-manager Helm Chart

dims opened this issue · 17 comments

dims commented

We have a helm chart which currently defaults to support for k8s 1.27. We do not have a CI job that tests this chart though.

Looking through usages in the wild:
https://github.com/search?type=code&q=%2Fhelm+repo+add+aws-cloud-controller-manager%2F&p=1

kops et al do use manifests directly:
https://github.com/search?q=repo%3Akubernetes%2Fkops+aws-cloud-controller-manager&type=code&p=4

Given community k8s 1.27 is EOL in June 2024 we should just drop the chart entirely by the time k8s 1.31 is released.

k8s-ci-robot commented

This issue is currently awaiting triage.

If cloud-provider-aws contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

kmala commented

/triage accepted

k8s-ci-robot commented

@kmala: The label triage/accepted cannot be applied. Only GitHub organization members can add the label.

In response to this:

/triage accepted

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

cartermckinnon commented

I think it's worth keeping around, it has a decent number of users (including Rancher: https://github.com/rancher/rke1-docs). Can we switch the values.yaml to use a latest tag/something low maintenance instead?

dims commented

@cartermckinnon we don't have a CI job to verify it does work .. so can't guarantee the functionality

atsai1220 commented

What would be the recommended way to install this cloud-provider?

atsai1220 commented
  • 👍1
et304383 commented

What would be the recommended way to install this cloud-provider?

https://github.com/kubernetes/cloud-provider-aws/blob/master/docs/getting_started.md#upgrading-an-existing-cluster

We are expected to apply a static example bunch of yaml files? They're still referencing version 1.27.1. What is the correct way to install the latest? Use this directory and do a bunch of sed commands for the latest?

Why not support the helm chart as the installation mechanism like the EBS CSI driver is doing? https://github.com/kubernetes-sigs/aws-ebs-csi-driver/releases

I hate to be blunt, but it's bad enough that the documentation for this CRITICAL project is severely lacking, but now a stable installation mechanism isn't even provided?

I REALLY wish AWS would chime in and take over this project so it has proper support instead of just expecting everyone to use EKS.

  • 👍6
dims commented

@et304383 can you please share where / how exactly you are using this chart?

et304383 commented

@et304383 can you please share where / how exactly you are using this chart?

We are managing our own K8s control plane for over 40 clusters, and obviously have to install this controller to support 1.27+

So far, this is how script to install:

helm repo add aws-cloud-controller-manager https://kubernetes.github.io/cloud-provider-aws
helm repo update
helm pull aws-cloud-controller-manager/aws-cloud-controller-manager --version 0.0.8 --untar

cd aws-cloud-controller-manager
cluster_cidr=$(grep cluster-cidr /etc/kubernetes/manifests/kube-controller-manager.yaml | cut -d = -f2)
cluster_name=$(grep cluster-name /etc/kubernetes/manifests/kube-controller-manager.yaml | cut -d = -f2)
yq -i '.args += ["--allocate-node-cidrs=false", "--configure-cloud-routes=false", "--cluster-cidr='$cluster_cidr'", "--cluster-name='$cluster_name'"]' values.yaml

export KUBECONFIG=/etc/kubernetes/admin.conf
helm upgrade --install -n kube-system aws-cloud-controller-manager .

You can see we already have to edit the values file to specify cluster cidr and cluster name (which there is zero documentation for, mind you), so adding another update to change the image version is just another line of code.

  • ❤️3
atsai1220 commented

We have been managing k8s cluster on ec2 instances with cloud-provider=external then installing this aws cloud-provider. We leverage the helm-controller inside k3s/rke2 to bootstrap the cluster with this cloud-provider.

apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: cloud-provider-aws
  namespace: kube-system
spec:
  chart: https://github.com/kubernetes/cloud-provider-aws/releases/download/helm-chart-aws-cloud-controller-manager-0.0.7/aws-cloud-controller-manager-0.0.7.tgz
  targetNamespace: kube-system
  bootstrap: true
  valuesContent: |-
    args:
      - --v=2
      - --cloud-provider=aws
      - --controllers=cloud-node,cloud-node-lifecycle,service,-route
      - '--cloud-config=/tmp/cloud-provider-config'
    nodeSelector:
      node-role.kubernetes.io/control-plane: "true"
    tolerations:
    - effect: NoSchedule
      key: ""
      operator: Exists
    - effect: NoExecute
      key: ""
      operator: Exists
    extraVolumes:
    - hostPath:
        path: /var/lib/rancher/rke2/etc/config-files/cloud-provider-config
        type: File
      name: cloud-provider-config
    extraVolumeMounts:
    - mountPath: /tmp/cloud-provider-config
      name: cloud-provider-config
apt-x4869 commented

We are managing multiple cluster. We need cloud provider helm chart to install for kubernetes cluster to work. We have automated process for upgrading and testing clusters so it is easier to use helm chart values to update , rather that manually updating in a yaml file. if something goes wrong in testing , it could be rolled back easily using helm.

I think its better to keep the helm chart.

#954

jobcespedes commented

Currently using helm chart for self managed clusters

sdlarsen commented

We're using it for self-managed clusters as well, specifying the tag in a values.yaml.