crossplane-contrib/provider-helm

ProviderConfig does not support "InjectedIdentity" as `source.identity`

Opened this issue · 0 comments

yogeek commented

What happened?

We have a GKE management cluster with crossplane 1.15.0 and we are writing a composition to provision managed GKE clusters along with addons services.

To achieve this, we followed the GCP upbound reference platform.
In this reference, the GKE composition creates a ServiceAccount and a ServiceAccountKey : https://github.com/upbound/configuration-gcp-gke/blob/main/apis/composition.yaml#L66-L89 to be used by both the GKE cluster and the Helm ProviderConfig.

However our company organization policy forbids the creation of a ServiceAccount Key : we are allowed to create ServiceAccount but not a ServiceAccountKey hence we cannot use the composition "as is".

Since #109 , it is possible to use identity in the Helm ProviderConfig (initially the need was to be able to use a ServiceAccount key in addition to the kubeconfig to authenticate to a GKE cluster).

In our case, as we cannot use a ServiceAccount key, we tried to use this identity field to reference a workloadIdentity for the ServiceAccount :

  • we created a ProjectIAMMember to add the IAM role roles/iam.workloadIdentityUser to the GCP ServiceAccount created by the composition
  • we configured the helm ControllerConfig with an annotation pointing to this GCP serviceAccount for the Helm provider
  • we added a new ProviderConfig with identity.source: InjectedIdentity (instead of referencing a secret containing a serviceaccount key)

In the end, we tried this :

(the ServiceAccount and the kubeconfig secret referenced below have already been created by the composition)

Provider 
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
name: provider-helm
spec:
controllerConfigRef:
  name: cgp-config-sa-tg
package: xpkg.upbound.io/crossplane-contrib/provider-helm:v0.15.0
ControllerConfig 
apiVersion: pkg.crossplane.io/v1alpha1
kind: ControllerConfig
metadata:
annotations:
  iam.gke.io/gcp-service-account: platform-ref-gcp-cluster@gcpXXXd-i3x5x6ms.iam.gserviceaccount.com
name: cgp-config-sa-tg
spec:
args:
- --debug
serviceAccountName: cgp-config-sa-tg
ProviderConfig 
apiVersion: helm.crossplane.io/v1beta1
kind: ProviderConfig
metadata:
name: platform-ref-gcp-cluster-test
spec:
credentials:
  secretRef:
    key: kubeconfig
    name: 80083b50-39fe-463c-9ad3-8765249f3c5e-gkecluster
    namespace: crossplane-system
  source: Secret
identity:
  source: InjectedIdentity                    # <<<<<<<<<<<<<<<<<<< this does not seem possible
  type: GoogleApplicationCredentials

Result :

  • a k8s serviceaccount cgp-config-sa-tg exists and is annotated with the GCP serviceaccount iam.gke.io/gcp-service-account: platform-ref-gcp-cluster@gcpXXXd-i3x5x6ms.iam.gserviceaccount.com
  • the helm provider pod is now using this cgp-config-sa-tg serviceaccount (so it should have the permision of the corresonding GCP IAM ServiceAccount, right...?)

But when we create a Release resource that references the platform-ref-gcp-cluster-test providerConfigRef, we get the following error :

2024-03-15T11:15:36.615Z	DEBUG	provider-helm	Cannot connect to provider	{"controller": "managed/release.helm.crossplane.io", "request": "/platform-ref-gcp-cluster-test", "uid": "a6e0e8cb-59df-42b9-888d-8f95d6ef1eea", "version": "10427210", "external-name": "platform-ref-gcp-cluster-test", "error": "failed to extract Google Application Credentials: no extraction handler registered for source: InjectedIdentity", "errorVerbose": "no extraction handler registered for source: InjectedIdentity\nfailed to extract Google Application Credentials\ngithub.com/crossplane-contrib/provider-helm/pkg/controller/release.(*connector).Connect\n\tgithub.com/crossplane-contrib/provider-helm/pkg/controller/release/release.go:190\ngithub.com/crossplane/crossplane-runtime/pkg/reconciler/managed.(*NopDisconnecter).Connect\n\tgithub.com/crossplane/crossplane-runtime@v0.19.2/pkg/reconciler/managed/reconciler.go:213\ngithub.com/crossplane/crossplane-runtime/pkg/reconciler/managed.(*Reconciler).Reconcile\n\tgithub.com/crossplane/crossplane-runtime@v0.19.2/pkg/reconciler/managed/reconciler.go:761\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.14.1/pkg/internal/controller/controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.14.1/pkg/internal/controller/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.14.1/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.14.1/pkg/internal/controller/controller.go:235\nruntime.goexit\n\truntime/asm_amd64.s:1594"}

2024-03-15T11:15:36.615Z	DEBUG	events	failed to extract Google Application Credentials: no extraction handler registered for source: InjectedIdentity	{"type": "Warning", "object": {"kind":"Release","name":"platform-ref-gcp-cluster-test","uid":"a6e0e8cb-59df-42b9-888d-8f95d6ef1eea","apiVersion":"helm.crossplane.io/v1beta1","resourceVersion":"10427210"}, "reason": "CannotConnectToProvider"}

it seems that Helm Provider is not supporting to have injectedIdentity inside the identity.source field : is that right ?

Is it something that needs to be implemented in the helm provider code or is there another solution we can use to achieve what we want without having to create a ServiceAccount key ?

What environment did it happen in?

Crossplane version: 1.15.0

  • Cloud provider : GCP
  • Kubernetes version : 1.27.8-gke
  • Kubernetes distribution : GKE