bug: bootstrap does not work in docker-from-docker setting
vkaiser-mb opened this issue · 6 comments
Are you use the envd server?
- Yes, I am using the envd server.
- No, I am not using the envd server.
Describe the bug
When trying to call envd bootstrap on my VScode devcontainer, I get this error message (last line is the error):
$ envd bootstrap
INFO[2023-10-30T13:04:38Z] [1/5] Bootstrap SSH Key
INFO[2023-10-30T13:04:38Z] [2/5] Bootstrap registry CA keypair
INFO[2023-10-30T13:04:38Z] [3/5] Bootstrap registry json config
INFO[2023-10-30T13:04:38Z] [4/5] Bootstrap autocomplete
INFO[2023-10-30T13:04:38Z] Install bash autocompletion
WARN[2023-10-30T13:04:38Z] Warning: failed writing to /usr/share/bash-completion/completions/envd: open /usr/share/bash-completion/completions/envd: permission denied
INFO[2023-10-30T13:04:38Z] You may have to restart your shell for autocomplete to get initialized (e.g. run "exec $SHELL")
INFO[2023-10-30T13:04:38Z] [5/5] Bootstrap buildkit
error: invalid mount config for type "bind": bind source path does not exist: /home/vscode/.config/envdI even tried to mount my local envd dir to the mentioned path and still get the same error.
To Reproduce
Devcontainer is in an docker-from-docker setting as described here:
https://github.com/microsoft/vscode-dev-containers/tree/main/containers/docker-from-docker
Expected behavior
No error
The docker info output
Client: Docker Engine - Community
Version: 24.0.6
Context: default
Debug Mode: false
Server:
Containers: 27
Running: 1
Paused: 0
Stopped: 26
Images: 90
Server Version: 24.0.6
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 nvidia runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 61f9fd88f79f081d64d6fa3bb1a0dc71ec870523
runc version: v1.1.9-0-gccaecfc
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: builtin
Kernel Version:
Operating System:
OSType: linux
Architecture: x86_64
CPUs:
Total Memory:
Name:
ID:
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
The envd version output
envd: v0.3.36
BuildDate: 2023-07-18T15:36:15Z
GitCommit: 854798c
GitTreeState: clean
GitTag: v0.3.36
GoVersion: go1.19.10
Compiler: gc
Platform: linux/amd64
Additional context
No response
Is it related to the https://github.com/microsoft/vscode-dev-containers/blob/main/containers/docker-from-docker/README.md#using-bind-mounts-when-working-with-docker-inside-the-container?
$HOME/.config/envd is mounted to share some configurations for buildkitd. You can avoid this by using a remote buildkitd or moby (no need to bootstrap a buildkit container):
- remote buildkitd
envd context create --name remote --builder tcp --builder-address <your-remote-buildkitd-addr> --use- moby (need to use the envd v1 syntax)
envd context create --name moby-test --builder moby-worker --useThanks that did the trick (option 2 with moby)!
Its OT, but now I face the issue that during the build its not using the proxy (it does when I use docker build). I guess I need to configure the proxy also for moby. In case you know this by hard, it would be awesome to get a little hint here, too.
Just out of curiosity, why is there no $HOME/.config/envd in my docker env after installing envd? And why does it say path does not exist, even when this exact path is mounted (from the host) and exists inside the container run?
envd up and envd build have the --proxy option that can use the HTTPS_PROXY/HTTP_PROXY environment variables.
That's because the $HOME/.config/envd is created in the "docker from docker" but the newly created container mounts the host instead of the "docker from docker" env. You can get the details from the previous link.
Thanks for help!
That helped, but again I get an proxy error:
#4 [internal] mkdir certs
#4 DONE 0.0s
#2 docker-image://docker.io/mambaorg/micromamba:1.0.0
#2 resolve docker.io/mambaorg/micromamba:1.0.0 2.4s done
#2 CACHED
#5 [internal] copy cert from mamba
#5 DONE 0.1s
#6 [internal] copy micromamba binary
#6 DONE 0.1s
#7 [internal] create envd python=3.11
#7 0.527 bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
#7 0.628
#7 0.628 __
#7 0.628 __ ______ ___ ____ _____ ___ / /_ ____ _
#7 0.628 / / / / __ `__ \/ __ `/ __ `__ \/ __ \/ __ `/
#7 0.628 / /_/ / / / / / / /_/ / / / / / / /_/ / /_/ /
#7 0.628 / .___/_/ /_/ /_/\__,_/_/ /_/ /_/_.___/\__,_/
#7 0.628 /_/
#7 0.628
#7 20.65 critical libmamba Multiple errors occured:
#7 20.65 Download error (28) Timeout was reached [https://repo.anaconda.com/pkgs/main/noarch/repodata.json]
#7 20.65 Could not resolve host: repo.anaconda.com
#7 20.65 Subdir pkgs/main/noarch not loaded!
#7 20.65 Subdir pkgs/r/noarch not loaded!
#7 20.65
#7 ERROR: process "bash -c /usr/local/bin/micromamba create -p /opt/conda/envs/envd -c defaults python=3.11" did not complete successfully: exit code: 1
------
> [internal] create envd python=3.11:
#7 0.628 / /_/ / / / / / / /_/ / / / / / / /_/ / /_/ /
#7 0.628 / .___/_/ /_/ /_/\__,_/_/ /_/ /_/_.___/\__,_/
#7 0.628 /_/
#7 0.628
#7 20.65 critical libmamba Multiple errors occured:
#7 20.65 Download error (28) Timeout was reached [https://repo.anaconda.com/pkgs/main/noarch/repodata.json]
#7 20.65 Could not resolve host: repo.anaconda.com
#7 20.65 Subdir pkgs/main/noarch not loaded!
#7 20.65 Subdir pkgs/r/noarch not loaded!If I would need to guess the proxy settings and env vars are not available during build. If you have another hint for me, that would be awsome.