databacker/mysql-backup

Issue using EKS IAM Roles to write to S3 bucket

pwatsonSANS opened this issue · 4 comments

We are using an EKS instance with containers configured using IAM service roles, so I am not setting the AWS key/secret env values

When the container tries to copy the backup to S3 it fails with the error

[Errno 13] Permission denied: '/home/appuser'

I opened a terminal to the container and made the aws call manually in debug mode
I can see a request/response to AWS to grab creds based on the role, but the process fails immediately after

MainThread - botocore.credentials - WARNING - Refreshing temporary credentials failed during mandatory refresh period.

I suspect that it is trying to cache the creds in the home/appuser folder, which doesn't exist so it fails

Is there a specific reason why the home folder is not created when creating the appuser user ?

I've not been able to get an example running locally that shows the problem to confirm

Changing the AWS_CONFIG_FILE and AWS_SHARED_CREDENTIALS_FILE env variables doesn't help either

deitch commented

I opened a terminal to the container and made the aws call manually in debug mode

As in using the aws CLI?

I can see a request/response to AWS to grab creds based on the role, but the process fails immediately after

The same error? And what happens if you create /home/appuser and chown it to be owned by that user, then try again?

yes - from within the contain i tried executing

aws s3 ls

and I get the same /home/appuser error as when i try to execute your container startup

I can't create the a home directory as I'm running under appuser and sudo isn't installed/no permissions to install

I'll build my own image with home folder and see if that works

Ok, adding the home directory fixes the issue i was having

RUN mkdir -p /var/cache/samba && chmod 0755 /var/cache/samba && chown appuser /var/cache/samba && chown appuser /var/lib/samba/private
RUN mkdir -p /home/appuser && chmod 0755 /home/appuser && chown appuser /home/appuser
USER appuser

Is working really nicely now thanks

PR created for the fix