AWS ECR credentials refresh with credentials helper
Closed this issue · 7 comments
What would you like to be added:
A possibility to automatically refresh the token when the initial token from AWS ECR credential helper is expired in 12 hours. falcoctl has the possibility to read the credentials from the AWS ECR credentials helper, and it works flawlessly. However, when the token is being expired, next call to AWS API by falcoctl won't be successful.
Why is this needed:
To follow a custom OCI artifact changes, published to the AWS ECR repo by falcoctl.
Hi @CarpathianUA, how did you configure falcoctl and AWS ECR credentials helper? falcoctl should automatically update the token when it expires.
Any info on how to reproduce the issue would be helpful.
Hi @alacuku!
I generate credStore string with init container for .docker/config.json, as well as download a docker-credential-ecr binary, since there is no other way to do that with a current falco Helm chart. Providing a static creds like a AWS access and secret key to falco-registry is not an option for me. Then I share a volume between containers in the falco pod. Here is a Helm values I use for falco chart to implement that:
...
extra:
initContainers:
- name: ecr-login
image: amazon/aws-cli:2.13.0
command: ["/bin/sh", "-c"]
args:
- |
token=$(aws ecr get-login-password --region us-east-1)
mkdir -p /root/.docker
echo -n '{"credsStore": "ecr-login", "auths": {" <REDACTED>.dkr.ecr.us-east-1.amazonaws.com": {"username": "AWS", "password": "'"$token"'", "email": "none"}}}' > /root/.docker/config.json
volumeMounts:
- name: docker-config
mountPath: /root/.docker
- name: docker-credential-ecr-binary
image: alpine:3.18
command: ["/bin/sh", "-c"]
args:
- |
apk update
apk add --no-cache curl
ARCH=$(uname -m)
VERSION="0.7.1"
if [ "$ARCH" = "aarch64" ]; then
URL="https://amazon-ecr-credential-helper-releases.s3.us-east-2.amazonaws.com/${VERSION}/linux-arm64/docker-credential-ecr-login"
else
URL="https://amazon-ecr-credential-helper-releases.s3.us-east-2.amazonaws.com/${VERSION}/linux-amd64/docker-credential-ecr-login"
fi
curl -so /mnt/shared-data/docker-credential-ecr-login "${URL}"
chmod +x /mnt/shared-data/docker-credential-ecr-login
volumeMounts:
- name: shared-data
mountPath: /mnt/shared-data
mounts:
# A list of volumes to add to the Falco pods.
volumes:
- name: docker-config
emptyDir: {}
- name: shared-data
emptyDir: {}
volumeMounts:
- name: docker-config
mountPath: /root/.docker
falcoctl:
artifact:
follow:
mounts:
volumeMounts:
- name: docker-config
mountPath: /root/.docker
- name: shared-data
mountPath: /usr/local/bin/docker-credential-ecr-login
subPath: docker-credential-ecr-login
env:
- name: PATH
value: /usr/local/bin:/mnt/shared-data
install:
mounts:
volumeMounts:
- name: docker-config
mountPath: /root/.docker
- name: shared-data
mountPath: /usr/local/bin/docker-credential-ecr-login
subPath: docker-credential-ecr-login
env:
- name: PATH
value: /usr/local/bin:/mnt/shared-data
config:
artifact:
install:
refs:
- falco-rules:1
- <REDACTED>.dkr.ecr.us-east-1.amazonaws.com/falco-rules:master
follow:
refs:
- falco-rules:1
- <REDACTED>.dkr.ecr.us-east-1.amazonaws.com/falco-rules:master
every: 3h
...
As was mentioned initially, falcoctl get the data from credStore successfully, and do rules OCI artifact pulls and pushes. But when the token is expired after 12h (defaults) next call to AWS API by falcoctl doesn't rotate a token from credStore.
Hi @CarpathianUA,
From your configuration
initContainers:
- name: ecr-login
image: amazon/aws-cli:2.13.0
command: ["/bin/sh", "-c"]
args:
- |
token=$(aws ecr get-login-password --region us-east-1)
mkdir -p /root/.docker
echo -n '{"credsStore": "ecr-login", "auths": {" <REDACTED>.dkr.ecr.us-east-1.amazonaws.com": {"username": "AWS", "password": "'"$token"'", "email": "none"}}}' > /root/.docker/config.json
volumeMounts:
- name: docker-config
mountPath: /root/.dockeryou are setting the credential manually in docker/config.json. Falcoctl will use those credentials but will not try to refresh them once they have expired. Please have a look here: https://github.com/awslabs/amazon-ecr-credential-helper#troubleshooting.
The correct setup would be to add the docker-credential-ecr-login binary to the falcoctl container, and then configure the AWS credentials and make them available to the credential helper: https://github.com/awslabs/amazon-ecr-credential-helper#aws-credentials.
The best solution would be to use IAM roles for service accounts and configure docker-credential-ecr-login accordingly.
Hi @alacuku , thank you for suggestions, I'll try them out.
Hi @CarpathianUA, any news?
Hi @alacuku,
Trying to get it works. So I've created an IAM role with appropriate policy to access ECR, assigned the role to the falco serviceAccount, added a docker-credential-ecr-login binary, and generated plain docker/config.json with just {"credsStore": "ecr-login"} in it.
What I get from falcoctl-artifact-install initContainer:
INFO: Installing the following artifacts: [ghcr.io/falcosecurity/rules/falco-rules:1 318522186253.dkr.ecr.us-east-1.amazonaws.com/falco-rules:master]
INFO: Preparing to pull "ghcr.io/falcosecurity/rules/falco-rules:1"
INFO: Retrieving credentials from local store
INFO: proceeding with empty credentials for registry "ghcr.io"
INFO: Pulling f6bc614bb812
INFO: Pulling 0780f3585aae
INFO: Pulling 1b155a3d8df8
INFO: Extracting and installing "rulesfile" "falco_rules.yaml.tar.gz"
INFO: Artifact successfully installed in "/rulesfiles"
INFO: Preparing to pull "<REDACTED>.dkr.ecr.us-east-1.amazonaws.com/falco-rules:master"
INFO: Retrieving credentials from local store
INFO: proceeding with empty credentials for registry "<REDACTED>.dkr.ecr.us-east-1.amazonaws.com"
ERRO: unable to fetch reference "<REDACTED>.dkr.ecr.us-east-1.amazonaws.com/falco-rules:master"
Are you sure that falcoctl-artifact-install can work this way? I mean with empty credentials in docker/config.json
@alacuku nevermind, all good. Thank you for your assistance!
So, to get it works what's need to be done:
.docker/config.jsonshould have{"credHelpers": {"<REDACTED>.dkr.ecr.<region>.amazonaws.com": "ecr-login"}}string and just it.- IAM policy for the role must have a dedicated term for:
...
{
Action = [
"ecr:GetAuthorizationToken"
]
Effect = "Allow"
Resource = "*"
},
...