Flux doesn't have permissions to access ECR while Kubelet uses the IAM Policies of the Node
Closed this issue · 2 comments
Describe the bug
Flux Image Automation Controller, Image-Reflector Controller, and Image Auto-Update resources do not have Permission to use the IAM Policies attached to the Node to pull and search for images from ECR.
It requires docker login to authenticate and uses a secret attached to a service account but the token expires after 12 hours. I have a working fix at the moment where I created a cronJob that renews the token right before it expires. But I don't see Kubelet using any CronJobs for deployments to access images on ECR. I believe this can be achieved with flux as well.
Steps to reproduce
- Install flux with the additional image automation controllers from the documentation
https://fluxcd.io/flux/guides/image-update/
- Follow the steps listed on the page for setting up the Image automation.
- create the imageRepository resource and attempt an image repo scan.
Expected behavior
The ImageRepository should be able to assume the AmazonEC2ContainerRegistryReadOnly policy I attached to the nodes that allow all workloads pull images from the container registry.
Screenshots and recordings
No response
OS / Distro
MacOS 15.1.1, But flux is on a cluster using linux nodes
Flux version
flux: v2.3.0
Flux check
► checking prerequisites
✗ flux 2.3.0 <2.4.0 (new CLI version is available, please upgrade)
✔ Kubernetes 1.30.6-eks-7f9249a >=1.28.0-0
► checking version in cluster
✔ distribution: flux-v2.3.0
✔ bootstrapped: true
► checking controllers
✔ helm-controller: deployment ready
► ghcr.io/fluxcd/helm-controller:v1.0.1
✔ image-automation-controller: deployment ready
► ghcr.io/fluxcd/image-automation-controller:v0.38.0
✔ image-reflector-controller: deployment ready
► ghcr.io/fluxcd/image-reflector-controller:v0.32.0
✔ kustomize-controller: deployment ready
► ghcr.io/fluxcd/kustomize-controller:v1.3.0
✔ notification-controller: deployment ready
► ghcr.io/fluxcd/notification-controller:v1.3.0
✔ source-controller: deployment ready
► ghcr.io/fluxcd/source-controller:v1.3.0
► checking crds
✔ alerts.notification.toolkit.fluxcd.io/v1beta3
✔ buckets.source.toolkit.fluxcd.io/v1beta2
✔ gitrepositories.source.toolkit.fluxcd.io/v1
✔ helmcharts.source.toolkit.fluxcd.io/v1
✔ helmreleases.helm.toolkit.fluxcd.io/v2
✔ helmrepositories.source.toolkit.fluxcd.io/v1
✔ imagepolicies.image.toolkit.fluxcd.io/v1beta2
✔ imagerepositories.image.toolkit.fluxcd.io/v1beta2
✔ imageupdateautomations.image.toolkit.fluxcd.io/v1beta2
✔ kustomizations.kustomize.toolkit.fluxcd.io/v1
✔ ocirepositories.source.toolkit.fluxcd.io/v1beta2
✔ providers.notification.toolkit.fluxcd.io/v1beta3
✔ receivers.notification.toolkit.fluxcd.io/v1
✔ all checks passed
Git provider
No response
Container Registry provider
Amazon ECR
Additional context
No response
Code of Conduct
- I agree to follow this project's Code of Conduct
In the ImageRepository
you need to set .spec.provider
to aws
, please see the docs here: https://fluxcd.io/flux/components/image/imagerepositories/#aws
In the
ImageRepository
you need to set.spec.provider
toaws
, please see the docs here: https://fluxcd.io/flux/components/image/imagerepositories/#aws
Thanks👍🏾