docker/compose

[BUG] Using up --pull always in combination with --no-build results in unexpected behaviour

domibarton opened this issue · 2 comments

domibarton commented

Description

The Docker compose behaviour when using the --no-build and --no-pull flags in combination will lead to unexpected errors.

Steps To Reproduce

Initial position

  • A Dockerfile
  • A Compose file with 2 services, namely:
    • A local service, which builds an image from the Dockerfile, without any image: tag
    • A remote service, which uses an image from a registry

Compose command

Run the following commands:

$ docker compose build
$ docker compose up -d --no-build --pull always

Expected behaviour

First command (build):

  • The image for the local service is built in the first command (i.e. build)
  • The image for the remote service is ignored, since it has not to be built

Second command (up):

  • The image for the remote service is pulled (because of --pull always)
  • The image for the local service isn't built (because of --no-build) or pulled (since there's no image: there)
  • Both containers are started

Actual behaviour

No issues with the first command.

For the second command, Docker tries to pull the local image without having an image: tag defined at all:

$ docker compose up -d --pull always --no-build
[+] Running 2/2
 ✘ local Error 
 ✔ remote Pulled 
Error response from daemon: pull access denied for docker-pull-test-local, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

More testing

Using only the --no-build flag works:

$ docker compose up -d --no-build
[+] Running 2/2
 ✔ Container docker-pull-test-local-1   Started
 ✔ Container docker-pull-test-remote-1  Started

Using only the --pull always flag works:

$ docker compose up -d --pull always
[+] Running 1/1
 ✔ remote Pulled
[+] Building 0.0s (0/0)
[+] Running 3/3
 ✔ Network docker-pull-test_default     Created
 ✔ Container docker-pull-test-local-1   Started
 ✔ Container docker-pull-test-remote-1  Started

Interestingly, --no-build --pull policy also fails:

$ docker compose up -d --pull policy --no-build
[+] Running 2/2
 ✘ local Error
 ✔ remote Pulled
Error response from daemon: pull access denied for docker-pull-test-local, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

Another bug

More interestingly, on another system, the --pull policy flag errors:

docker compose up -d --pull policy --no-build
invalid --pull option "policy"

The help states it should be working (it is on the other system):

 docker compose up -h
Flag shorthand -h has been deprecated, please use --help

Usage:  docker compose up [OPTIONS] [SERVICE...]

Create and start containers

Options:
~snip~
      --pull string               Pull image before running ("always"|"policy"|"never") (default "policy")
~snap~

Environment

I tried to run this on:

  • Debian 12 (amd64)
  • Docker version 24.0.7, build afdd53b
  • Docker Compose version v2.21.0

The system where the --pull policy fails, is this:

  • macOS 13.5.2 (arm)
  • Docker version 24.0.6, build ed223bc
  • Docker Compose version v2.23.0-desktop.1

Compose Version

Docker version 24.0.7, build afdd53b
Docker version 24.0.6, build ed223bc

Docker Environment

Client: Docker Engine - Community
 Version:    24.0.7
 Context:    default
 Debug Mode: false
 Plugins:
  compose: Docker Compose (Docker Inc.)
    Version:  v2.21.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 20
  Running: 18
  Paused: 0
  Stopped: 2
 Images: 32
 Server Version: 24.0.7
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f
 runc version: v1.1.10-0-g18a0cb0
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.1.0-13-amd64
 Operating System: Debian GNU/Linux 12 (bookworm)
 OSType: linux
 Architecture: x86_64
 CPUs: 20
 Total Memory: 62.6GiB
 Name: nyx
 ID: 9768c347-9421-401b-adba-f3f88e36b5f2
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.30.0.0/16, Size: 28

Anything else?

I've created a repo to demonstrate it: https://github.com/confirm/docker-pull-test

domibarton commented

It gets weirder

Running this command sequentially will not change the running container:

$ docker compose up -d --no-build local
[+] Running 1/0
 ✔ Container docker-pull-test-local-1  Running 

$ docker compose up -d --no-build local
[+] Running 1/0
 ✔ Container docker-pull-test-local-1  Running
 
$ docker compose up -d --no-build local
[+] Running 1/0
 ✔ Container docker-pull-test-local-1  Running

However, changing then the parameters, the container is recreated:

$ docker compose up -d --pull always local
[+] Running 1/1
 ✔ Container docker-pull-test-local-1  Started

Why is it recreated? :/

Funnily enough, now running the same command again results in no recreation again:

$ docker compose up -d --pull always local
[+] Running 1/0
 ✔ Container docker-pull-test-local-1  Running

$ docker compose up -d --pull always local
[+] Running 1/0
 ✔ Container docker-pull-test-local-1  Running

$ docker compose up -d --pull always local
[+] Running 1/0
 ✔ Container docker-pull-test-local-1  Running

Switching CLI args means recreate every time:

$ docker compose up -d --no-build local
[+] Running 1/1
 ✔ Container docker-pull-test-local-1  Started 

$ docker compose up -d --pull always local
[+] Running 1/1
 ✔ Container docker-pull-test-local-1  Started

$ docker compose up -d --no-build local
[+] Running 1/1
 ✔ Container docker-pull-test-local-1  Started 

$ docker compose up -d --pull always local
[+] Running 1/1
 ✔ Container docker-pull-test-local-1  Started

Hint: It's the «interactive progress bar». When it states Running, nothing was recreated. However, when it shows Started, it was Recreate & Recreated before.

ndeloof commented

please don't mix issues this make it harder to diagnose and ensure we don't miss anything

about --pull policy you've been confused by a mistake in the docs (already fixed). Supported values are missing and always