How to `docker-compose up --force-recreate` without reusing previous volumes

[hdorio]

With docker-compose 1.4.2 and docker 1.8.2
You can see below the first volume e583c6a8...5a93788a0 is reused

 $ sudo docker-compose up -d --force-recreate
   Recreating remotetransmission_torrent_1...

 $ docker inspect remotetransmission_torrent_1 | grep volumes
   "/mnt/docker/volumes/e583c6a87437a5b4b1af50ee2693bd3e5dce574ec72d60dce1311215a93788a0/_data:/home/transmission/.config/transmission-daemon:rw",
   "/mnt/docker/volumes/cefce79850d7162f4f99541559c2dfc7315c83db717a7a5953118bd3c4b273e0/_data:/home/transmission/Downloads:rw"
   "Source": "/mnt/docker/volumes/e583c6a87437a5b4b1af50ee2693bd3e5dce574ec72d60dce1311215a93788a0/_data",
   "Source": "/mnt/docker/volumes/cefce79850d7162f4f99541559c2dfc7315c83db717a7a5953118bd3c4b273e0/_data",

 $ sudo docker-compose up -d --force-recreate
   Recreating remotetransmission_torrent_1...

 $ docker inspect remotetransmission_torrent_1 | grep volumes
   "/mnt/docker/volumes/e583c6a87437a5b4b1af50ee2693bd3e5dce574ec72d60dce1311215a93788a0/_data:/home/transmission/.config/transmission-daemon:rw",
   "/mnt/docker/volumes/cefce79850d7162f4f99541559c2dfc7315c83db717a7a5953118bd3c4b273e0/_data:/home/transmission/Downloads:rw"
   "Source": "/mnt/docker/volumes/e583c6a87437a5b4b1af50ee2693bd3e5dce574ec72d60dce1311215a93788a0/_data",
   "Source": "/mnt/docker/volumes/cefce79850d7162f4f99541559c2dfc7315c83db717a7a5953118bd3c4b273e0/_data",

I'm forced to stop then rm to create some new volumes

 $ sudo docker-compose stop 
   Stopping remotetransmission_torrent_1... done

 $ sudo docker-compose rm
   Going to remove remotetransmission_torrent_1
   Are you sure? [yN] y
   Removing remotetransmission_torrent_1... done

 $ sudo docker-compose up -d --force-recreate
   Creating remotetransmission_torrent_1...

 $ docker inspect remotetransmission_torrent_1 | grep volumes
   "Source": "/mnt/docker/volumes/c5bb9a8f7b68c762c42e9c0ee92afbca3aa0d7ff9d09aaf45fd260f6fc663ec9/_data",
   "Source": "/mnt/docker/volumes/9dcce8440bafc8893e07352111d1aefb625c36df10da6dc4eaa593220266ea31/_data",

_
Is there a better way than the stop/rm method?

[dnephin]

stop/rm is the right way. The data in volumes could be important, so we want to make it difficult to accidentally remove it.

We could probably document this a little better.

[MicahZoltu]

This policy seems to violate docker best practices, and has caused us tons of headache troubleshooting why recreating containers was leaking state. When I tell docker to recreate, that doesn't mean "persist some data across the container runs but restart the processes in the containers". It means pave the earth and start over. If I wanted to save the volumes I would explicitly mount volumes. I never expect auto-mounted volumes of any kind to persist across container runs.

[dnephin]

Persisting data across runs is really the only reason to use volumes. If you don't want persisted data why are you putting it in a volume?

[MicahZoltu]

I'm not setting up a volume. My docker-compose.yml doesn't have any volumes setup and I'm not passing anything to docker-compose.yml to attach any volumes.

Command:

docker-compose up --force-recreate --abort-on-container-exit --build foo

docker-compose.yml:

version: '2'
services:
  foo:
    build:
      context: .
      dockerfile: src/integration/foo/Dockerfile
    ports:
      - "3306:3306"
      - "33060:33060"

Dockerfile:

FROM mysql:5.7

COPY schema/foo/migration.sql /data/db_schema.sql
COPY src/integration/foo/create_test_db.sh /docker-entrypoint-initdb.d/create_test_db.sh
ENV MYSQL_ALLOW_EMPTY_PASSWORD true

EXPOSE 3306 33060

create_test_db.sh:

#!/bin/bash
set -e
mysql --no-defaults -u root -e "drop database if exists agent_state; create database foo"
mysql --no-defaults -u root foo < "/data/db_schema.sql"

If I run the above, write some stuff to the DB, then SIG_INT, then run the command again the data I put in the DB is persisted across the run.

[dnephin]

That's an issue with the MySQL image. It creates a volume in the base
image. You can work around the problem by either using a different MySQL
image or possibly by forcing it to use a different path for the data.

On Oct 19, 2016 6:36 PM, "Micah Zoltu" notifications@github.com wrote:

I'm not setting up a volume. My docker-compose.yml doesn't have any
volumes setup and I'm not passing anything to docker-compose.yml to
attach any volumes.

Command:

docker-compose up --force-recreate --abort-on-container-exit --build foo

docker-compose.yml:

version: '2'
services:
foo:
build:
context: .
dockerfile: src/integration/foo/Dockerfile
ports:
- "3306:3306"
- "33060:33060"

Dockerfile:

FROM mysql:5.7

COPY schema/foo/migration.sql /data/db_schema.sql
COPY src/integration/foo/create_test_db.sh /docker-entrypoint-initdb.d/create_test_db.sh
ENV MYSQL_ALLOW_EMPTY_PASSWORD true

EXPOSE 3306 33060

create_test_db.sh:

#!/bin/bash
set -e
mysql --no-defaults -u root -e "drop database if exists agent_state; create database foo"
mysql --no-defaults -u root foo < "/data/db_schema.sql"

If I run the above, write some stuff to the DB, then SIG_INT, then run the
command again the data I put in the DB is persisted across the run.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#2127 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAa_RG_pJj0i-OSCfcBlG__8ToFDtGKMks5q1sWlgaJpZM4GHruC
.

[MicahZoltu]

Hmm, this violates my understanding of docker containers. How is it mounting a volume without me providing a path on the host? My understanding is that docker containers are ephemeral unless you explicitly mount a volume?

[dnephin]

A volume doesn't need a host path. There are three kinds of volumes:

• host bind mounts (-v /host:/container)
• named volumes (-v name:/container, created with docker volume create)
• anonymous volumes (-v /container, or VOLUME inside a Dockerfile)

The mysql image uses an anonymous volume. At runtime you can tell the container to use a different volume for that path in the container, but if you don't the anonymous volume is still there.

Anonymous volumes are not great. They are the oldest of the three, and much of their behaviour is legacy stuff that is only kept for backwards compatibility reasons.

[monotek]

At least some command line option a la "--recreate-volumes" would be nice...

[hleb-rubanau]

OK, guys, here's the case to think of:

1. I want to remount my Rails public directory to nginx container, so that some static stuff could be served by nginx directly.
2. I assign an "anonymous" (not host-mounted, not named) volume to /usr/local/app/app/public, and share it to nginx via "volumes_from"
3. "public" contents is often changed on image builds -- not only top-level files but some files in subdirectories (this is important)

Currently, if I recreate containers, I end up with old version of "public" -- contents in the image is simply ignored. Yes, Docker is supposed to copy missing files into the anonymous volume, but for variety of reasons it does not always happen (I suspect, there's no deep check of subdirectories structure).

So, I am either forced to perform "stop-rm-up" sequence (not that convenient in production), or to use separate directory as a shared volume, and explicitly call 'rsync' on application container start to populate/update it.

If there was a way to just let anonymous volume go along with the parent container, it would be great improvement.

[dnephin]

Don't use volumes for code (or static assets). Volumes are for data that you want to preserve between deployments, which is the opposite of what you want here. Either build the nginx image with the static assets, or proxy some web server container that contains them.

[hleb-rubanau]

Thanks for the viewpoint! I did not thought from this perspective yet.

It seems to be one of conceptual problems with (still evolving) Docker architecture. We saw evolution of data volumes concept so far (e.g. from "data containers" to "named volumes") and probably it is not yet finished.

If you'll look at https://docs.docker.com/engine/tutorials/dockervolumes/#/data-volumes , you'll see that most described benefits (bypassing AUFS, sharing) are not necessarily connected to data persistence (which volumes were originaly designed for).

So, no surprise people (including me) are trying to use volumes in a variety of ways beyond their original purpose. E.g. to share ephemeral or image-controlled data from one container to another, without lots of explicit copying.

Maybe some day we'll figure out the consistent standard way to do it. :) So far, it is possible to use some fairly simple workarounds described above. Not ideal, but acceptable, as long as architectural expectations are properly set.

Once again, thanks for your response, clarification and for the great work you are doing!

[brthor]

I ran into this issue today, thanks for the great explanations here everyone, has definitely helped me understand the problem.

A major point of confusion for me was understanding that the VOLUME tag in a dockerfile causes a consistent anonymous volume to be created. I may have missed it in the documentation but couldn't find mention of it.

[brunocascio]

@dnephin

As mentioned @hleb-rubanau. Running single container with Rails and Nginx is the solution? (Like this: https://docs.docker.com/engine/admin/multi-service_container/ ?)

Should I break the best practices ("Each container should have only one concern
") just for servings assets? :(

[hleb-rubanau]

For those interested, I ended up with the following:

1. In my setups I always use bind-mounted (aka host-mounted) volumes now. Anonymous and named volumes have too much special/unobvious/conditional/inconsistent/impicit logic to take into account. Yes, it is easy to understand the specifics of each type's lifecycle and management, but I found it's easier for me to think about architecture when I am not concerned about all those irrelevant differences. For distributed FS Gluster works fine (still being bind-mounted from Docker perspective).

2. Shared volume is bind-mounted one, and it is not the directory where assets are stored in the image. At the start (in the entrypoint) I am running local rsync, so that assets from image's directory are copied/rsynced to the bind-mounted shared storage path.

[brunocascio]

Thanks @hleb-rubanau

Anyway, I'm not sure if some orchestration tools like RancherOS supports those volume configurations or is the right way in order to scale. At the end, is more difficult deploy with docker...

I prefer to use anonymous volumes and clean the orphans at some point.

This is my production rails stack: https://github.com/brunocascio/AR-MTB/blob/master/docker-compose.prod.yml

[ashb]

This also caused me a large amount of confusion when I was testing the jenkins/jenkins image and it wasn't respecting my changes from the files in /usr/share/jenkins/ref because it had already copied them.

That is a pretty unexpected user experience - docker compose is creating a "hidden" volume for all intents and purposes. If you use normal docker run then you get a new volume each time. It's a hidden volume in the sense that nothing in the docker-compose up talks about it, so unless you know in detail what the image is doing you have to go digging.

At the very least we should print a message saying "not re-creating volume x" so that other people in the future don't have to waste time wondering what is happening.

[namgivu]

Hi @dnephin,

I call sudo rm -rf /var/lib/docker/volumes/aa_dbdatavol and now I cannot docker-compose up my postgres docker-compose any more.

Getting this error

Creating network "aa_default" with the default driver
Creating aa_postgres_1
ERROR: for postgres Cannot create container for service postgres: no such file or directory
ERROR: Encountered errors while bringing up the project.
Error response from daemon: No such container: aa_postgres_1

As you may have some idea to get around that, please share. Thank you!

[namgivu]

#2127 (comment)

Just re-creating the folder will fix my issue.
/var/lib/docker/volumes/aa_dbdatavol/_data

[settermjd]

Big thanks to @dnephin, your answer worked perfectly, the first time. Muchos gracias.

[omidkrad]

@dnephin said:

Persisting data across runs is really the only reason to use volumes. If you don't want persisted data why are you putting it in a volume?

to share files between containers. Is there a better way to do that without using volumes?

[dnephin]

What do you mean by "share files"?

Are you expecting that one container will write to a file, and the other container will see those writes? A file system is generally not a good interface between two services, but if that is the case you could have one of the services "manage" the filesystem by writing updates to the volume.

If "sharing" is just that two containers happen to read some of the same files then there is no need for a volume. Add the file to both containers with COPY.

[mauricios]

I'm using Docker version 18.06.1-ce, build e68fc7a and I can use the following command to recreate anonymous volumes:

docker-compose up -d --build --force-recreate --renew-anon-volumes db

It seems the flag --renew-anon-volumes was recently added

[dbjpanda]

I'm using Docker version 18.06.1-ce, build e68fc7a and I can use the following command to recreate anonymous volumes:

docker-compose up -d --build --force-recreate --renew-anon-volumes db

It seems the flag --renew-anon-volumes was recently added

Thanks for letting this option.