Support `cp' sub command
arnisoph opened this issue · 33 comments
For now, docker-compose cp is missing. :)
$ docker-compose -v
docker-compose version 1.7.1, build 0a9ab35
$ docker-compose -h
Define and run multi-container applications with Docker.
Usage:
docker-compose [-f <arg>...] [options] [COMMAND] [ARGS...]
docker-compose -h|--help
Options:
-f, --file FILE Specify an alternate compose file (default: docker-compose.yml)
-p, --project-name NAME Specify an alternate project name (default: directory name)
--verbose Show more output
-v, --version Print version and exit
-H, --host HOST Daemon socket to connect to
--tls Use TLS; implied by --tlsverify
--tlscacert CA_PATH Trust certs signed only by this CA
--tlscert CLIENT_CERT_PATH Path to TLS certificate file
--tlskey TLS_KEY_PATH Path to TLS key file
--tlsverify Use TLS and verify the remote
--skip-hostname-check Don't check the daemon's hostname against the name specified
in the client certificate (for example if your docker host
is an IP address)
Commands:
build Build or rebuild services
config Validate and view the compose file
create Create services
down Stop and remove containers, networks, images, and volumes
events Receive real time events from containers
exec Execute a command in a running container
help Get help on a command
kill Kill containers
logs View output from containers
pause Pause services
port Print the public port for a port binding
ps List containers
pull Pulls service images
restart Restart services
rm Remove stopped containers
run Run a one-off command
scale Set number of containers for a service
start Start services
stop Stop services
unpause Unpause services
up Create and start containers
version Show the Docker-Compose version information
@bechtoldt What should cp do?
docker-compose cp <container> <path> <local path>, e.g.
$ docker-compose cp db /var/lib/postgresql ~/Downloads/database_backup
That would be an awesome feature.
On top of that a support of copy in the configuration file (#1664) would be perfect
+1 for this feature. need copy some files into running container
+1.... having to build a container to get a large file or 2 updates is a pain.
As a workaround, you can do:
docker cp /path/to/my-local-file.sql "$(docker-compose ps -q mycontainer)":/file-on-container.sql
Now /file-on-container.sql will exist on your running container:
docker-compose exec mycontainer /bin/bash -c 'ls -lah /file-on-container.sql'
-rw-r--r-- 1 root root 244M Nov 14 19:48 /file-on-container.sql
I use this bash alias:
alias docker-compose-cp="function _f(){ docker cp \$2 \`docker-compose ps -q \$1\`:\$3;};_f"
:)
With a "files" option in Compose which can copy local file to remote container between creation and starting process, it could be amazing to inject config files and secrets 👍
With local docker daemon, you can use "volumes" to map local file in a container, but it is not possible with remote docker daemon.
version: '2'
services:
service1:
image: nginx
files:
- nginx.conf:/etc/nginx/conf.d/nginx.conf
- cert.crt:/etc/nginx/certs/cert.crt
The workarounds suggested here are not bad, but the prime reason to want this is so that the cp is guaranteed to be executed before the command, which like @johnmarcou points out allows docker-compose to interact much better with remote docker daemons (i.e. ANY cloud based CI service). This should be a high priority feature imo.
Wouldn't it make more sense to use the service name rather than the container name ?
docker-compose cp [OPTIONS] SERVICE SRC_PATH DEST_PATH
So that we're in sync with the syntax of docker-compose exec which is:
$ docker-compose exec
Execute a command in a running container
Usage: exec [options] SERVICE COMMAND [ARGS...]
How would it work if there are multiple instances of the same service?
If there are multiple containers, which one should it copy to?
Was there any resolution to this? I think this would be a very nice feature!
Currently not. I'm using docker cp still. Somewhat inconvenient since you always need to know the ID or the name of the container.
I think this feature will be great!!
I'll try to implement this command.
Is this topic still alive? It would be great to be able to use 'cp' with service tags instead of container name (which can be dynamic).
Sorry. I was doing other things. 😅
I restarted implementing recently.
If anyone else is trying this, I'll respect them.
Any update on this? Ran into a use case where this would have been helpful.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
This is not stale, still relevant and necessary
This issue has been automatically marked as not stale anymore due to the recent activity.
See also #5523 (and the comment containing my workaround)
As demonstrated there, the docker-compose ps -q workaround doesn't work for the use case of copying files out of a container because you get the ID of the container before any artifacts were created.
2nd of June 2020 now, (almost) two entire years after the first proposal...
Any news about replacing "a whack ton of time and space to build one image per configuration file and deploying 27000 different images" with "just a copy command in your compose yaml"?
I would need to deploy a demo of a service, with that scope in mind, I need to use a sqlite3 file instead of keeping everything on memory to allow me to use known admin credentials and not let people administer whatever they want (even in the demo image).
That poses a challenge: how I reset the database every X hours if not by keeping a local copy of the base image with just the database file as difference, and using the local build (or image) in docker-compose when I just had to change that file (perfect scope for docker-compose cp on the base image after up, and just reload the service to pick up the new config and db files)?
@bryanpedini It's my understanding that your use case is a docker anti-pattern. Docker containers are intended to be ephemeral. In other words, configuration management should be managed at either startup, as part of the entry point scripts, or done real time (with caching) with APIs to a configuration management services (AWS SSM parameter store, spring configuration server, etc, etc).
@TrentonAdams while I agree that @bryanpedini 's particular case isn't 100% "docker-esque" ... there are still use cases where this would be convenient... For example, copying a small number of small files (e.g. crypto keys) into the image at startup. Sure, could use a volume or even build a custom image but those options (while certainly docker-esque) seem like such overkill for 2 or 3 ~5kb files. Especially if you want to customize on a per-image basis.
Further, with docker-compose, if you're copying in the files at each start in the yaml... that's still ephemeral. (I guess it's kinda like a "mini" layer at the end of the container image at that point...)
@bryanpedini Perhaps you can "docker cp" into the containers created by docker-compose?
@danieldjewell That sounds like the job of a startup script. e.g. enabling an env var that has data that gives permission to a crypto service, then connecting to said crypto service and grabbing keys for your needs. In the case of AWS for example, your container would receive an IAM access key for a role, which has permissions to use AWS KMS.
Or, you can put those keys (text only) themselves into ENV vars (not secure), if you're not using such services. Or mount the files. Or if you're using pure docker, it has facilities to share secret data. Mounting files, using docker's secrets, or using a configuration service at startup, is usually the best option.
@TrentonAdams My understanding (which can be totally biased/wrong, not native Eng speaker here) of the situation is that Docker was not "meant" for this type of thing, tho IMO it's so much convenient to administer an image/container instead of an entire service on the host machine.
That said, I resolved my demo "issue" by having a custom image with just config file and SQLite3 DB file with credentials (otherwise empty) on top of the base image; then the docker-compose YAML uses the custom img instead of the baseline; and finally a cronjob that resets the compose stack (literally one container with one custom image).
That, if more "Docker-esque", is quite less convenient than just a cronjob with cd /opt/<service>; docker-compose cp <srvc_name> <file>; docker-compose restart <srvc_name>...
That's all 😅
Docker was not "meant" for this type of thing
Not meant for what type of thing?
There's a really good use case for this. In a CI system, I want to run tests in a docker container and get a coverage report. The actual docker host is a remote machine so I can't use simple volume mounts. Therefore, the only way to get the coverage report is to docker cp the file from the container onto the CI node.
docker compose cp is implemented by Compose v2
sorry, I'm not all informed on the current news about everything in IT, but from my understanding what is now docker-compose as a separate binary and set of tools that has to be installed/updated/managed separately, will in the future be a docker <subcommand> like ps or rmi, integrated and installed/updated/managed with the single docker binary itself?
compose v2 is indeed a docker - actually a CLI plugin (separated binary) but will be distributed with the docker CLI. For now it's still beta so need to be installed in addition to the docker CLI, see https://github.com/docker/compose-cli#compose-v2-aka-local-docker-compose