Support named volumes

[aanand]

Currently, Compose will treat the part of a volumes entry before the : (if present) in two different ways:

• If you've specified a volume_driver, it will treat it as a volume name and pass it straight through to the Docker API.
• Otherwise, it will treat it as a filesystem path and perform expansion of relative paths and the ~ character.

However, Docker will eventually be adding support for volumes as a top-level concept (docker volumes), and therefore implicitly for named local volumes - e.g. a volume named data which refers to an arbitrary filesystem path on the host machine, which you can mount with docker run -v data:/data.

The Docker daemon decides whether to treat the value as a path or a volume name based on whether it's an absolute path, which on Unix means it starts with a slash and on Windows is, uh, complicated. But we've got an extra challenge because Compose allows relative paths and ~. We need a way of deciding whether or not to perform path expansion before passing the value to the API.

We have a number of options. In increasing order of my personal preference:

1. Try passing the value straight through to the API. If it fails, perform expansion and try again. I don't like this because it complicates the client-side flow a lot (we can't know what the right value is until it's time to create a container) and requires multiple API calls.

2. Perform expansion on the value, see if it's a valid local path, and use that if so; otherwise, pass it straight through. I don't like this because it means we'll do completely different things if you misspell a path, forget to create it etc.

3. Treat the value as a path if it contains (or perhaps starts with) any of .~/. This is simple enough, but will break backwards compatibility. Entries like this one will now be treated differently:

   volumes:
     - data:/data

   Where this previously meant "mount the data subdirectory of the current directory", it would now mean "mount the volume named data".

   If we go down this route, it might be best to put out a deprecation warning in the next release whenever we see host paths that don't start with one of the special characters, so users have a release cycle to update their YAML (i.e. to change data to ./data), and then make the breaking change in the subsequent release.

   If we do that, we need to do it now - 1.4 will be out very soon.

Thoughts?

[thaJeztah]

Related, but no direct solution, I think Docker automatically creating the mount point / directories if it didn't exist was a bad decision. I.e. docker run -v /non-existing-path:/foo .. should fail instead of creating /non-existing-path; this would've allowed 2. and still being consistent with docker.

I like 3 (and a deprecation message now), but worried what ./ means in compose; right now, all paths are relative to the yaml file, which makes it great, because the projects are "portable". Will that still be possible with the new paths (using ./)?

[aanand]

Yes, ./ will still mean "the directory in which this current Compose file is located".

[aanand]

Implemented option 3 in #1833.

[dave-tucker]

👍 for option 3

[Silex]

Hum, I can't figure what "named volumes" are from the doc:

http://docs.docker.com/engine/userguide/dockervolumes/

The container-dir must always be an absolute path such as /src/docs. The host-dir can either be an absolute path or a name value. If you supply an absolute path for the host-dir, Docker bind-mounts to the path you specify. If you supply a name, Docker creates a named volume by that name

That's the only reference to named volumes I could find, and it doesn't explain what it is.

[thaJeztah]

@Silex docker now has volume management commands, which allow you to manage volumes even if there's no container using them anymore. Regular / unnamed volumes get a random "UUID" as name, but by setting a name here, you can reference the volume by that name.

Also see;

• https://docs.docker.com/engine/reference/commandline/volume_create/
• https://docs.docker.com/engine/reference/commandline/volume_ls/
• https://docs.docker.com/engine/reference/commandline/volume_inspect/
• https://docs.docker.com/engine/reference/commandline/volume_rm/

But we may need to more explicitly explain "named" volumes in that context

[Silex]

@thaJeztah: thanks for the info!

[thaJeztah]

@Silex you're welcome!

If you have suggestions on how to improve this in the documentation, we welcome pull requests! The Markdown file used to generate that page can be found in the docker/docker repository; https://github.com/docker/docker/blob/master/docs/userguide/dockervolumes.md

Otherwise, please open an issue; https://github.com/docker/docker/issues so that we don't loose sight