fluxcd/flux2

Flux doesn't have permissions to access ECR while Kubelet uses the IAM Policies of the Node

Closed this issue · 2 comments

Describe the bug

Flux Image Automation Controller, Image-Reflector Controller, and Image Auto-Update resources do not have Permission to use the IAM Policies attached to the Node to pull and search for images from ECR.
It requires docker login to authenticate and uses a secret attached to a service account but the token expires after 12 hours. I have a working fix at the moment where I created a cronJob that renews the token right before it expires. But I don't see Kubelet using any CronJobs for deployments to access images on ECR. I believe this can be achieved with flux as well.

Steps to reproduce

  1. Install flux with the additional image automation controllers from the documentation https://fluxcd.io/flux/guides/image-update/
  2. Follow the steps listed on the page for setting up the Image automation.
  3. create the imageRepository resource and attempt an image repo scan.

Expected behavior

The ImageRepository should be able to assume the AmazonEC2ContainerRegistryReadOnly policy I attached to the nodes that allow all workloads pull images from the container registry.

Screenshots and recordings

No response

OS / Distro

MacOS 15.1.1, But flux is on a cluster using linux nodes

Flux version

flux: v2.3.0

Flux check

► checking prerequisites
✗ flux 2.3.0 <2.4.0 (new CLI version is available, please upgrade)
✔ Kubernetes 1.30.6-eks-7f9249a >=1.28.0-0
► checking version in cluster
✔ distribution: flux-v2.3.0
✔ bootstrapped: true
► checking controllers
✔ helm-controller: deployment ready
► ghcr.io/fluxcd/helm-controller:v1.0.1
✔ image-automation-controller: deployment ready
► ghcr.io/fluxcd/image-automation-controller:v0.38.0
✔ image-reflector-controller: deployment ready
► ghcr.io/fluxcd/image-reflector-controller:v0.32.0
✔ kustomize-controller: deployment ready
► ghcr.io/fluxcd/kustomize-controller:v1.3.0
✔ notification-controller: deployment ready
► ghcr.io/fluxcd/notification-controller:v1.3.0
✔ source-controller: deployment ready
► ghcr.io/fluxcd/source-controller:v1.3.0
► checking crds
✔ alerts.notification.toolkit.fluxcd.io/v1beta3
✔ buckets.source.toolkit.fluxcd.io/v1beta2
✔ gitrepositories.source.toolkit.fluxcd.io/v1
✔ helmcharts.source.toolkit.fluxcd.io/v1
✔ helmreleases.helm.toolkit.fluxcd.io/v2
✔ helmrepositories.source.toolkit.fluxcd.io/v1
✔ imagepolicies.image.toolkit.fluxcd.io/v1beta2
✔ imagerepositories.image.toolkit.fluxcd.io/v1beta2
✔ imageupdateautomations.image.toolkit.fluxcd.io/v1beta2
✔ kustomizations.kustomize.toolkit.fluxcd.io/v1
✔ ocirepositories.source.toolkit.fluxcd.io/v1beta2
✔ providers.notification.toolkit.fluxcd.io/v1beta3
✔ receivers.notification.toolkit.fluxcd.io/v1
✔ all checks passed

Git provider

No response

Container Registry provider

Amazon ECR

Additional context

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct

In the ImageRepository you need to set .spec.provider to aws, please see the docs here: https://fluxcd.io/flux/components/image/imagerepositories/#aws

In the ImageRepository you need to set .spec.provider to aws, please see the docs here: https://fluxcd.io/flux/components/image/imagerepositories/#aws

Thanks👍🏾