An official Docker Image with the AWS CLI for use in CI/CD scenarios
matti opened this issue · 121 comments
I read Issue #3529 and #3291 and saw those were closed, with the only reaction hinting it was 'not that complicated'. However, the comment also acknowledged that doing this yourself would run the risk of being out of date. Apart from exactly that point, I would also like to point out that, for commercial users, having an official Amazon image is hugely preferential to "/aws-cli:latest".
In my case, I would be using this in a Google Cloud Build because it is far superior than AWS CodeBuild.
I am here because I too am looking for an official aws-cli docker image to use in my CI/CD environment. Instead I am now going to create ANOTHER pipeline to build a docker image which includes the aws cli when I could just reference the official one in my original pipeline.
Also...someone else gets it too https://hub.docker.com/r/google/cloud-sdk.
Isn't +1 considered harmful ;)
Anyway this is the third (?) issue created about this...
👍 add my +1
I have to create my own docker image to only include awsCLI in my CI. I would prefere an official one
+1
+1
+1
+1
+1
+1
+1
+1
+1
+1
Heres an alpine (post fix) image with the most recent cli installed for anyone else looking for a recent version in the mean time.
@justnance enough?
+1
+1
+1
+1
Another reason is when using an OS like coreOS that has no package management the idiomatic way of running things is via a container. I'm surprised the need for this would even be questioned, is an obvious omission. 👍
+1
👍
+1
+1
+1
+1
+1
As the opener of #3291 (the earliest of the three quoted Issues lol), I'm glad to see that this issue is finally gaining traction. To all future responders, when @justnance says "Please +1 it to show interest", he means add a like on the first comment, that's the proper way to +1 things on GitHub so that you're not spamming the maintainers' mailboxes.
+1 to keep repo owners notified
+1
When they say to +1 an issue, what they mean is to click the 👍 button. It doesn't help the developers to have 100 comments that each say "+1". The more you know!
@davidham - Thanks for clarifying and everyone for responding. I second that. Please use the GitHub reactions and click the 👍 button.
I said the same thing 2 days ago but hey we're all on the same side 🙃
+1
+1
+1
+1
+1
+1
+1
Can you stop +1? if you want to +1, do it on the top.
You are wasting valuable engineer time. We are dozen subscribed to this issue...
I have been maintaining an aws-cli image for over two years now. Feel free to use it if needed (I use it several times a day). I receive updates on new version releases (via IFTT) and push updates fairly quickly. Despite the fame and glory of running my own image (ha!), I will gladly defer and push folks to use an official image.
After having used my image for a long time, I will say that it would be super helpful if the official image included jq (as the API is JSON heavy). It does make it super convenient to do things such as "grab latest ECS task definition, make a change, and push it back" all in a single CI/CD pipeline stage.
Yet another alternative solution to the problem: https://hub.docker.com/r/somedeveloper/aws-cli/
Reasons for using:
- Keeps it simple.
- Uses official python3.7-stretch as base image.
- Uses AWS recommended strategy for installation -- python + pip. see here.
- Includes aws-sam-cli for those Serverless nerds 8-).
- Its public and doesn't require a login.
- Great for CI/CD pipelines -- haven't used it for anything else so haven't weighed the pros and cons.
Still hoping for an official image. Think about the people now
https://hub.docker.com/r/google/cloud-sdk/ . Sorry, guys. It is such a hard task to do for a giant like AWS.
+1
Isn't +1 considered harmful ;)
Anyway this is the third (?) issue created about this...
Fourth, if you consider amazonlinux/container-images#10.
Can't we just put it in CircleCI and be done with it? Happy to help with the Dockerfile or pipeline.
I wonder if there are any internal restrictions and/or paperworks inside Amazon that keeps the developers away from accomplishing this seemingly trivial work...
k lol
There are a few variants that would be nice to have in an "official" image. For instance, I'm currently looking to grab a container with the aws cli and curl (to check the IAM metadata endpoint) and it would be really handy to find one I could just grab and plug into my k8s deployment.
There is also a security reason for providing official images.
It makes the threat model simpler by removing a dependency on a "random person on the internet" maintaining the image that is used in high value targets like CI/CD pipelines that usually give a lot of access to your systems.
I would like an official aws-cli docker image based from the docker:18 (current stable) image (https://hub.docker.com/_/docker) - eg. aws-cli-docker-18 because I would like to build my docker image in my CI/CD environment that currently uses docker:18 image and publish it to AWS ECR.
+1
+1
It would be great if people would refrain from commenting when their comment does not contribute to the issue at hand. Comments such as "+1" just introduce spam for subscribers and make the issue lengthier than necessary. Instead, give a thumbs up to the first comment of the issue.
It would be great if people would refrain from commenting when their comment does not contribute to the issue at hand. Comments such as "+1" just introduce spam for subscribers and make the issue lengthier than necessary. Instead, give a thumbs up to the first comment of the issue.
This issue has been open since September of last year. I think we need to ask AWS to take a look at this issue again, since there is obviously interest in it. We should be told how much interest is "enough".
since there is obviously interest in it
Not just interest, but is the opened issue with more 👍 reactions:
https://github.com/aws/aws-cli/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc
The 2nd with most reactions was opened in 2014 and the 3rd in 2015, while this issue was opened in September of 2018 (less than a year ago).
+1000
I have on my local machine all the time issues with setting up the right dependencies to make aws-cli work. Therefore I decided to switch to aws-cli inside docker. I found several publicly available images on docker-hub BUT because they are not official I don't trust them by default so every time there is an updated version available I have to search and find again an image which can be trusted. and is secure to use. Please AWS build, maintain and provide a secure aws-cli docker image via docker hub. Thanks in advance
There's a huge fragmentation of community provided aws-cli images. But, as mentioned earlier: people would prefer one that's officially maintain and supported by Amazon. Several reasons why:
- Won't stale out -- community images are notorious for eventually going stale;
- More secure -- it's from a trusted resource, no matter how trusted a community member may be, they don't officially represent Amazon, therefore "all warranties void";
- Official support means official support in the event of bugs, version conflicts, backwards compatibilities, etc.
- AWS can update their image as they update the
awscli, including historical tags.
+1
+1 It's really unfortunate that Amazon does not provide this
I have since added another image that has been handy. It combines Docker in Docker with the AWS and SAM CLI which makes for a perfect ECR integration.
+1
Whilst there are loads of unofficial images out there that are well maintained, what's to stop them one day from saying "Pfft, what's the point of updating this anymore. This is Amazon's job!"
+1
+1
When it comes to helping people automate their work with AWS Amazon are failing big time, this is one of the reasons we are thinking of leaving them.
Is there an official response from maintainers regarding whether or not this is on a road map yet? Like many others, I would prefer to use an official image as well...
+1
If there were an official image for the AWS CLI it would be an executable sitting inside a scratch container. Because that wouldn't be incredibly useful for individuals what might be best is to:
- Use an official python container to build the CLI from source
- Copy the resulting AWS CLI binary into a busybox or scratch container
Anything more would be over-engineering despite any debates occurring here.
AWS loves to demo all of it's ECR and CodeDeploy services. Why won't they point all that firepower at their own containerized CLI?
Because the offer on the table it seems is for someone from the community to maintain it.
@balibebas somebody from the community is already doing it. There are a ton of images out there - but the point here is that there would be one by AWS, because I don't want to run randomguy/aws-cli:canbechangedanytime in my CI environment with all the secrets wide open.
What would you say about F-Droid then?
it is as relevant to this issue as this cat:
You sound like a paying customer.
well maybe that is because I am
Preaching to the choir. Anyway the Brew formula had a better chance because it's been trolled longer, still with no movement. So cat photos become immediately counter productive when there's no general use case outside maybe a scratch or busybox container as I pointed out earlier. What's your design proposal?
me and many others are currently doing "busybox containers" for example to run it with docker to get ECR auth credentials. given how many docker things aws has, it makes no sense that an official packaging does not exist.
Maybe they're just onto something else. ;)
I'm glad we got this sorted out now. Back to +1 comments:
+1 for Dockerless
@matti @balibebas Keep in mind that as of right now there are 64 people in this issue's conversation alone, and every response triggers an unhelpful email and notification that potentially gets sent to every one of those 64 people and more who are subscribed. This message will do the same, and I apologize for that.
But really, please keep things professional. While cat pictures are nice, the further this thread goes off the rails I feel the less likely it is to be taken seriously by the maintainers. Unfortunately, spam is spam is spam.
That's what the unsubscribe button does, the one I just clicked.
That would be a good thing to have. I am only surprised that no action is being taken or this is being maintained (includes being said NO).
Anyway people above have rightly pointed out the importance of an official aws cli image.
I think people are already using their own privately or via somebody else's built image / package managers route etc.
Yet Another Example Script for the same
But the simplest & non-avoidable problem remains the same.
- The never ending list of dependencies & uncertainties that hops in if it depends on developers to integrate things on their own. Will eventually lead to more issues in the repo as people will start with installing one thing then other and then a few other to get work done contaminating the environment which defeats the purpose of CI/CD or similar works of being isolated & pristine.
- Its hard to trust to implement & track anybody else's docker image for your work. This leads to making another pipeline & entry addition in docker registry for new awscli image, another repository i.e another thing to maintain.
- In CI/CD, my preference is to just use the image (our internal or official) & to avoid scripting lines to add things as much as possible.
- Problem with build & libs as official image can be right away build from source releases itself & have less dynamic dependencies wrt package manager & what not.
else
=> Everybody ends up making their own.
same here, currently i use a self build image, but would prefer a official image under the
namespace. I use it to build other docker images and push them to ECR, where the awscli is needed to get the credentials.
FROM docker:18.06
RUN apk update && \
apk -Uuv add python py-pip && \
pip install awscli && \
apk --purge -v del py-pip && \
rm /var/cache/apk/*
It shouldn't be that big deal to add this to your awscli build pipeline ... :)
--
i have updated the Dockerfile according to @mikesir87 suggestion, thx for that (thats another reason to have a standard image with contributions of the comunity to get the smallest image)
Sorry to spam everyone, but wanted to point out (in case anyone else wants to use the snippet from @jens-meiss) that you should really consider chaining your three RUN statements into a single statement. Otherwise, you're still shipping around the contents of py-pip and the apk caches in the intermediate layers, even though your final container can't access them.
On another note, the comment brings up another valid use case... when you're using the aws-cli only to get credentials for ECR to push images. This sounds like a need for packaging up the ECR credential helper in an image too. It would certainly make it easy to build and push images without needing the full aws-cli.
Hi everyone, maintainer here. Just wanted to clarify, this is happening, we're doing this.
We're in the process of building out better release infrastructure internally to be able to build/test/support more types of release artifacts, especially as we'll be shipping more release artifacts for cli v2.
We don't have an exact timeline at the moment, but it is happening.
Easy thing to do according to amazon devs and many others, yet still waiting after 2 years and no ETA 😂
+1
Internal releasing infrastructure (2019 Q4)
legal team assessment (2020 Q1)
public beta under aws/cli-dev-test (2020 Q2)
final release (2020 Q3)
In this optimistic timeline, you will get what you want in less than 10 months. 🥇
waiting for jeff barrs blog post
I'm damn waiting for this.
Can we at least get an official announcement or commitment. Maybe a target release ?
@bhmckendrick is a commitment not exactly what this comment not too far above yours is?
over a year old and no updates? Image?
hey @jamesls, would you consider hard-locking this thread until you have something to share?
the number of people wholly unwilling to read (hint hint) and instead choosing to spam 70+ watchers about how they specifically are annoyed is, I'm sure, drastically lowering everyone's interest in following this thread.
also, thanks for your work towards making this happen!
As the original author of this issue (well, I was just brave enough to copy/paste the previously closed "wontfix" ones) I have already unsubscribed from this issue because of the massive +1 spam and occasional fights with cat pictures (sorry).
Just adjust notification settings so that you only get notifications if this issue is closed.
I'd just like to point out that aside from CI/CD, some developers (see @LongLiveCHIEF), prefer to have dockerized development environments, and don't like installing things natively, or having to deal with the ensuing version managers for the native languages those cli tools inevitably rely on.
It's easier to docker pull aws-cli than whatever the existing install steps are... not to mention if you're not a python developer, you have the overhead of setting up a good python version and environment for your user, or perhaps even each project.
Scale that to all the different tools a developer might use (ruby based cli's, node based cli's), and you have to learn environment setups for languages you might not ever code in.
The point I'm making here is that docker run is ubiquitous, and gets rid of any native language setup/configs, and makes things easy for users.
Even if users build their own docker images, they still have to struggle with those setup tasks.
I don't code in python, but I've been forced to learn the ins and outs of virtual environments and their best practices from various versions of python, purely because tool providers consider it "trivial".
Not all developers have the same background as those who built the tool, and providing a docker image is a sign of respect. The tool providers can take on the overhead of native language specific environment quirks very quickly and easily, whereas adopters have to spend far more time to manage this overhead through the various lifecycle stages of development with your product.
Food for though.
@jamesls Thanks for listening to community feedback here. While waiting for the official hosted docker images, a helpful intermediate step might be to post install recommendations for a few popular base docker images here (eg. node, alpine, ubuntu, amazon2, python). This would be immediately valuable to us.
As I work for big ass enterprise let me serve you with this:
https://github.com/aws/aws-codebuild-docker-images
https://hub.docker.com/r/amazon/aws-codebuild-local