Specifing `docker --config` option breaks `docker buildx build` command
yankeguo opened this issue · 4 comments
Contributing guidelines
- I've read the contributing guidelines and wholeheartedly agree
I've found a bug and checked that ...
- ... the documentation does not mention anything about my problem
- ... there are no open or closed issues that are related to my problem
Description
I use different DOCKER_CONFIG directories to switch among different tenants of the same Container Registry.
Recently, I found that when I set the DOCKER_CONFIG directory parameter, whether through the --config parameter or the DOCKER_CONFIG environment variable, the docker buildx build command parameter parsing will be completely messed up, reporting that --platform, -t and other parameters cannot be recognized.
Expected behaviour
Command like
docker --config /path/to/custom-config-dir buildx build --platform linux/amd64 -t myimage1 -t myimage2 --build-arg APP_NAME=server --push .should work
Actual behaviour
An error raised:
unknown flag: --platform
See 'docker --help'.
Usage: docker [OPTIONS] COMMAND
A self-sufficient runtime for containers
Common Commands:
run Create and run a new container from an image
exec Execute a command in a running container
...
When I omit --config option, everything works.
Even if I set DOCKER_CONFIG environment variable, it will also fall.
Buildx version
github.com/docker/buildx v0.16.2-desktop.1 081c21b9e461293ae243a1ff813a680a4f5f8fb9
Docker info
Client:
Version: 27.2.0
Context: desktop-linux
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.16.2-desktop.1
Path: /Users/myusername/.docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.29.2-desktop.2
Path: /Users/myusername/.docker/cli-plugins/docker-compose
debug: Get a shell into any image or container (Docker Inc.)
Version: 0.0.34
Path: /Users/myusername/.docker/cli-plugins/docker-debug
desktop: Docker Desktop commands (Alpha) (Docker Inc.)
Version: v0.0.15
Path: /Users/myusername/.docker/cli-plugins/docker-desktop
dev: Docker Dev Environments (Docker Inc.)
Version: v0.1.2
Path: /Users/myusername/.docker/cli-plugins/docker-dev
extension: Manages Docker extensions (Docker Inc.)
Version: v0.2.25
Path: /Users/myusername/.docker/cli-plugins/docker-extension
feedback: Provide feedback, right in your terminal! (Docker Inc.)
Version: v1.0.5
Path: /Users/myusername/.docker/cli-plugins/docker-feedback
init: Creates Docker-related starter files for your project (Docker Inc.)
Version: v1.3.0
Path: /Users/myusername/.docker/cli-plugins/docker-init
sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
Version: 0.6.0
Path: /Users/myusername/.docker/cli-plugins/docker-sbom
scout: Docker Scout (Docker Inc.)
Version: v1.13.0
Path: /Users/myusername/.docker/cli-plugins/docker-scout
Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 27.2.0
Storage Driver: overlayfs
driver-type: io.containerd.snapshotter.v1
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 8fc6bcff51318944179630522a095cc9dbf9f353
runc version: v1.1.13-0-g58aa920
init version: de40ad0
Security Options:
seccomp
Profile: unconfined
cgroupns
Kernel Version: 6.10.4-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: aarch64
CPUs: 8
Total Memory: 7.655GiB
Name: docker-desktop
ID: 9b936eee-5f78-458b-adf1-6c115f5a42e0
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Labels:
com.docker.desktop.address=unix:///Users/myusername/Library/Containers/com.docker.docker/Data/docker-cli.sock
Experimental: false
Insecure Registries:
hubproxy.docker.internal:5555
127.0.0.0/8
Live Restore Enabled: false
WARNING: daemon is not using the default seccomp profile
Builders list
-> % docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS
default docker
\_ default \_ default running v0.15.2 linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64
desktop-linux* docker
\_ desktop-linux \_ desktop-linux running v0.15.2 linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64
Configuration
FROM alpine:3.20
ARG APP_NAME="server"
ENV TZ="Asia/Shanghai"
RUN apk add --no-cache tini ca-certificates tzdata curl && \
ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && \
echo $TZ > /etc/timezone
ENTRYPOINT [ "/sbin/tini", "--" ]
WORKDIR /data
ADD ${APP_NAME} /${APP_NAME}
RUN chmod +x /${APP_NAME}
ENV APP_NAME="${APP_NAME}"
CMD [ "sh", "-c", "exec /${APP_NAME}" ]Build logs
No response
Additional info
No response
@thaJeztah I can't reproduce but the error printed here("A self-sufficient runtime for containers") is coming from docker/cli
I suspect the problem is in this part;
buildx: Docker Buildx (Docker Inc.)
Version: v0.16.2-desktop.1
Path: /Users/myusername/.docker/cli-plugins/docker-buildx
CLI plugins are discovered in a number of locations; some "system-wide", and some "per user"
/usr/[local/]lib/docker/cli-plugins/usr/[local/]libexec/docker/cli-plugins~/.docker/cli-plugins/
The last one allows overriding the system-installed version, but I think the ~/.docker here is not hard-coded, but "config-directory". So using --config <custom directory> not only switches the location to look for the config-file, but also other content related to the config-directory (including those plugins, and likely contexts and state stored by plugins inside the config-directory).
Trying to mimic that situation;
docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock docker:cli
apk add -qq --no-cache js
docker info --format '{{json .ClientInfo.Plugins}}' | jq .
[
{
"SchemaVersion": "0.1.0",
"Vendor": "Docker Inc.",
"Version": "v0.17.1",
"ShortDescription": "Docker Buildx",
"Name": "buildx",
"Path": "/usr/local/libexec/docker/cli-plugins/docker-buildx"
},
{
"SchemaVersion": "0.1.0",
"Vendor": "Docker Inc.",
"Version": "v2.29.7",
"ShortDescription": "Docker Compose",
"Name": "compose",
"Path": "/usr/local/libexec/docker/cli-plugins/docker-compose"
}
]Moving the plugins to ~/.docker/cli-plugins;
mkdir -p ~/.docker/cli-plugins/
mv usr/local/libexec/docker/cli-plugins/* ~/.docker/cli-plugins/Without custom config-dir;
docker build --help
Start a build
Usage: docker buildx build [OPTIONS] PATH | URL | -
Start a build
...With a custom config-dir;
docker --config=/foo/ build --help
DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
Install the buildx component to build images with BuildKit:
https://docs.docker.com/go/buildx/
Usage: docker build [OPTIONS] PATH | URL | -This is somewhat of a known issue; there's various tickets (and backlog items) related to revamping storage locations (also in relation to rootless mode, which has some additional logic, and use of XDG directories). While it's a known issue, it's grown to be more problematic with the introduction of many plugins, and other parts of docker (ab)using ~/.docker for a growing list of purposes (which can be particularly problematic when that involves platform-specific content, and users (e.g.) bind-mount the config-directory into a container).
It's not a trivial thing to change though, as there's many tools that implemented these paths and logic, so at least it will require a long transition period; it will also involve changes in Docker Desktop (which uses this location for various purposes as well, including cli-plugins it ships).
Let me at least transfer this ticket to the CLI repository, which has the core parts of this logic, and link it from the internal backlog item(s) related to this.