error exporting image / operating system is not supported with deployments/k3d
KaiWalter opened this issue · 26 comments
When running make up from folder /deployments/k3d
...
docker buildx build --platform=wasi/wasm --load -t wasmtest_spin:latest ../../images/spin
[+] Building 0.5s (18/18) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 726B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/golang:1.19.5-bullseye 0.4s
=> [internal] load metadata for docker.io/library/rust:1.67 0.4s
=> [build-go 1/5] FROM docker.io/library/golang:1.19.5-bullseye@sha256:78b1726316ebfdec022fe7628f07547cafcd2d8a3934288187b87173f75925bc 0.0s
=> [build 1/4] FROM docker.io/library/rust:1.67@sha256:aed98ad355d0e8303d057cc4894a7d331010980eb63c7dc46176de82670881dd 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 307B 0.0s
=> CACHED [build 2/4] WORKDIR /opt/build 0.0s
=> CACHED [build 3/4] COPY . . 0.0s
=> CACHED [build 4/4] RUN rustup target add wasm32-wasi && cargo build --target wasm32-wasi --release 0.0s
=> CACHED [stage-2 1/3] COPY --from=build /opt/build/target/wasm32-wasi/release/spin_rust_hello.wasm . 0.0s
=> CACHED [stage-2 2/3] COPY --from=build /opt/build/spin.toml . 0.0s
=> CACHED [build-go 2/5] WORKDIR /opt/build 0.0s
=> CACHED [build-go 3/5] COPY . . 0.0s
=> CACHED [build-go 4/5] RUN curl -LO https://github.com/tinygo-org/tinygo/releases/download/v0.25.0/tinygo_0.25.0_amd64.deb && dpkg -i tinygo_0.25.0_amd64.deb 0.0s
=> CACHED [build-go 5/5] RUN cd go-hello && tinygo build -wasm-abi=generic -target=wasi -gc=leaking -o spin_go_hello.wasm main.go 0.0s
=> CACHED [stage-2 3/3] COPY --from=build-go /opt/build/go-hello/spin_go_hello.wasm . 0.0s
=> ERROR exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:67311f8ddbc767ceb889f36a141e8333b1ccd0f40aa4144716967e6b2db2e07f 0.0s
------
> exporting to image:
------
ERROR: failed to solve: operating system is not supported
make: *** [Makefile:29: build-workload-images] Error 1
OS
$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
Docker
$ docker version
Client: Docker Engine - Community
Version: 23.0.3
API version: 1.42
Go version: go1.19.7
Git commit: 3e7cbfd
Built: Tue Apr 4 22:03:23 2023
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 23.0.3
API version: 1.42 (minimum version 1.12)
Go version: go1.19.7
Git commit: 59118bf
Built: Tue Apr 4 22:03:23 2023
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.20
GitCommit: 2806fc1057397dbaeefbea0e4e17bddfbd388f38
runc:
Version: 1.1.5
GitCommit: v1.1.5-0-gf19387a
docker-init:
Version: 0.19.0
GitCommit: de40ad0
$ docker buildx version
github.com/docker/buildx v0.10.4 c513d34
I get the same error when following this sample https://nigelpoulton.com/getting-started-with-docker-and-wasm/ - hence I need to continue looking what is missing in my setup
$ docker buildx build --platform wasi/wasm -t docker-wasm:0.1 .
[+] Building 0.2s (5/5) FINISHED
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 156B 0.0s
=> [internal] load .dockerignore 0.1s
=> => transferring context: 2B 0.0s
=> [internal] load build context 0.1s
=> => transferring context: 2.11MB 0.0s
=> [1/1] COPY ./target/wasm32-wasi/release/hello-docker.wasm /hello-docker.wasm 0.0s
=> ERROR exporting to image 0.1s
=> => exporting layers 0.0s
=> => writing image sha256:0798ce03a89761749e66262a7bc9e330a10ffc634074f9ad34084f6d5235bedf 0.0s
------
> exporting to image:
------
ERROR: failed to solve: operating system is not supported
OK, I'm building out all the deployments and can confirm that something's up!!!
I’m on a flight most of today, but I’ll try to take a look at the airport / in flight.
Thank you guys for looking into this.
So far I tried on Linux Ubuntu+Debian with Docker Engine 23.x without an idea how to enable https://docs.docker.com/desktop/wasm/ --> https://docs.docker.com/desktop/containerd/.
I will now try with Windows WSL 2 / Docker Desktop 4.18.0 with the Features in development / Use containerd for pulling and storing images enabled.
installing containerd is easy enough -- it's the wasi/wasm buildx step that seems to blow chunks.
trying to understand the buildx version and interaction:
ERROR: failed to solve: operating system is not supported
22429 0.10.3+azure-1 /usr/libexec/docker/cli-plugins/docker-buildx -D buildx build --platform=wasi/wasm --load -t wasmtest_slight:latest ../../images/slight
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryClient.func1.1.1
/build/moby-buildx/.gopath/src/github.com/docker/buildx/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:72
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc.UnaryClientInterceptor.func1
/build/moby-buildx/.gopath/src/github.com/docker/buildx/vendor/go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc/interceptor.go:99
github.com/moby/buildkit/client.filterInterceptor.func1
/build/moby-buildx/.gopath/src/github.com/docker/buildx/vendor/github.com/moby/buildkit/client/client.go:294
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryClient.func1.1.1
/build/moby-buildx/.gopath/src/github.com/docker/buildx/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:72
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryClient.func1
/build/moby-buildx/.gopath/src/github.com/docker/buildx/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:81
google.golang.org/grpc.(*ClientConn).Invoke
/build/moby-buildx/.gopath/src/github.com/docker/buildx/vendor/google.golang.org/grpc/call.go:35
github.com/moby/buildkit/api/services/control.(*controlClient).Solve
/build/moby-buildx/.gopath/src/github.com/docker/buildx/vendor/github.com/moby/buildkit/api/services/control/control.pb.go:2208
github.com/moby/buildkit/client.(*Client).solve.func2
/build/moby-buildx/.gopath/src/github.com/docker/buildx/vendor/github.com/moby/buildkit/client/solve.go:258
golang.org/x/sync/errgroup.(*Group).Go.func1
/build/moby-buildx/.gopath/src/github.com/docker/buildx/vendor/golang.org/x/sync/errgroup/errgroup.go:75
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1594
22429 0.10.3+azure-1 /usr/libexec/docker/cli-plugins/docker-buildx -D buildx build --platform=wasi/wasm --load -t wasmtest_slight:latest ../../images/slight
github.com/moby/buildkit/client.(*Client).solve.func2
/build/moby-buildx/.gopath/src/github.com/docker/buildx/vendor/github.com/moby/buildkit/client/solve.go:273
golang.org/x/sync/errgroup.(*Group).Go.func1
/build/moby-buildx/.gopath/src/github.com/docker/buildx/vendor/golang.org/x/sync/errgroup/errgroup.go:75
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1594
@KaiWalter do docker buildx version and report out what you have on both build machines if you would.
OK, so update: for whatever reason, the buildx version I'm using -- we need to check yours -- doesn't seem to like the --platform string in that build. If I modify it to wasm it builds FINE:
docker buildx build --platform=wasm --load -t wasmtest_slight:latest ../../images/slight
[+] Building 1.7s (13/13) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/rust:1.64 1.6s
=> [build 1/5] FROM docker.io/library/rust:1.64@sha256:53ded1c919ea0dc23be959e28037238d8c321cd88fbac04d818f210031fccc3b 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 357B 0.0s
=> CACHED [build 2/5] WORKDIR /opt/build 0.0s
=> CACHED [build 3/5] COPY . . 0.0s
=> CACHED [build 4/5] RUN rustup target add wasm32-wasi && cargo build --target wasm32-wasi --release 0.0s
=> CACHED [build 5/5] RUN apt-get update && apt-get install ca-certificates -y 0.0s
=> CACHED [stage-1 1/3] COPY --from=build /opt/build/target/wasm32-wasi/release/http_server_lib.wasm ./app.wasm 0.0s
=> CACHED [stage-1 2/3] COPY --from=build /opt/build/slightfile.toml . 0.0s
=> CACHED [stage-1 3/3] COPY --from=build /etc/ssl /etc/ssl 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:58fd3cb85426cca89b90756652792f86cb9cbb5e1bcbf0d1ae2cba45f6cdb106 0.0s
=> => naming to docker.io/library/wasmtest_slight:latest
Ubuntu:
github.com/docker/buildx v0.10.4 c513d34
Windows (socketed to WSL2):
github.com/docker/buildx v0.10.4 c513d34049e499c53468deac6c4267ee72948f02
Windows inside WSL2 / Ubuntu:
github.com/docker/buildx v0.10.4 c513d34049e499c53468deac6c4267ee72948f02
I am building buildx v0.10.4 on ubuntu as well. try grabbing that docker buildx build string, and modifying the platform to just wasm and see if it goes through.
With my Windows / WSL2 combination it worked
~/src/containerd-wasm-shims/deployments/k3d$ make test
curl localhost:8081/spin/hello
Hello world from Spin!curl localhost:8081/slight/hello
I think it has something to do with this switch
- I'm glad that it works for you using the tools Docker has put together.
- We're going to have to understand what needs to be done to enable this in moby buildx as well. That's clearly the issue. I've just upgraded buildx and have the same thing. I'll look into the containerd settings as well.
Hi everyone,
The issue here is the docker daemon, it's not related to buildx/buildkit, buildx manages to build the image but dockerd returns an error because it can't store wasm images.
@KaiWalter from your docker version output I can see that on linux you have the 23.0.3 dockerd version, this one doesn't have the needed code for you to run wasm workloads.
We don't have a stable release with all of the containerd image store code needed for Wasm yet but you could try it out with the 24.0.0-beta.1 that has the containerd image store feature. You can download it from https://download.docker.com/linux/static/test/x86_64/docker-24.0.0-beta.1.tgz. Once downloaded you will need to edit the /etc/docker/daemon.json file (create it if it doesn't exist) and make sure the file contains the "features" property:
{
"features": {
"containerd-snapshotter": true
}
}And then you need to:
- Run the daemon with
sudo ./dockerd -D -H unix:///tmp/docker.sock --data-root /tmp/dockerd --pidfile /tmp/docker.pid - Create a context that will connect to this daemon:
docker context create test --docker "host=unix:///tmp/test.sock" - Use that context:
docker context use test - Run a wasm workload with docker or run
make up
Once you are done testing Wasm you can go back to your original context:docker context use default
This will all be easier on linux once 24.0 is released (soon).
On Windows/Linux/Mac this all should work once you check the User containerd for pulling and storing feature in Docker Desktop.
Cheers @squillace and @rumpl
Legends!
one of us is a legend. One of us is just old. Very different.
@rumpl I followed your instructions above for Linux (Debian 11), needing to run dockerd and make up as sudo, but still get this message:
=> ERROR exporting to image 1.4s
=> => exporting layers 0.0s
=> => exporting manifest sha256:6de5f0ecd3e0ee17cee51cc25bb696eead01959ac29eaef6fb8f13c05efb6aca 0.0s
=> => exporting config sha256:508159fa344c5f41864a21b47d6646161cdddfd8ae49feb20ad74a330596d624 0.0s
=> => exporting attestation manifest sha256:50912c607e292b9f092e2162b3213371ebafc78b923877a7f32dbab23910c306 0.5s
=> => exporting manifest list sha256:7685195d70a615fa350cfaf6b26d21e8ab173480bfdc13260c66200ba02a0126 0.3s
=> => naming to docker.io/library/wasmtest_spin:latest 0.1s
=> => unpacking to docker.io/library/wasmtest_spin:latest 0.0s
------
> exporting to image:
------
ERROR: failed to solve: no match for platform in manifest sha256:7685195d70a615fa350cfaf6b26d21e8ab173480bfdc13260c66200ba02a0126: not found
make: *** [Makefile:29: build-workload-images] Error 1
Anything else I miss?
I'm going to try the beta 2 release. But as @rumpl told me, there are a few more prs to get in before the DD functionality goes to moby.
installed beta-2 with
curl -fsSL https://test.docker.com -o test-docker.sh
sudo sh test-docker.sh
/etc/docker/daemon.json:
{
"features": {
"containerd-snapshotter": true
}
}
docker info
...
Server Version: 24.0.0-beta.2
Storage Driver: overlayfs
driver-type: io.containerd.snapshotter.v1
created a wasi/wasm builder for buildx:
docker buildx create --name wasm-builder --platform wasi/wasm,linux/amd64
changed deployments/k3d/Makefile - had to fall back to image file save + load as image loading did not work
...
TEST_IMG_NAME_SPIN ?= wasmtest_spin:latest
TEST_IMG_NAME_SPIN_FILE ?= ~/wasmtest_spin.tar
TEST_IMG_NAME_SLIGHT ?= wasmtest_slight:latest
TEST_IMG_NAME_SLIGHT_FILE ?= ~/wasmtest_slight.tar
...
build-workload-images:
docker buildx use wasm-builder
docker buildx build --platform=wasi/wasm --load -t $(TEST_IMG_NAME_SPIN) ../../images/spin
docker buildx build --platform=wasi/wasm --load -t $(TEST_IMG_NAME_SLIGHT) ../../images/slight
docker buildx use default
load-workload-images: build-workload-images
docker save $(TEST_IMG_NAME_SPIN) -o $(TEST_IMG_NAME_SPIN_FILE)
k3d image load $(TEST_IMG_NAME_SPIN_FILE)
docker save $(TEST_IMG_NAME_SLIGHT) -o $(TEST_IMG_NAME_SLIGHT_FILE)
k3d image load $(TEST_IMG_NAME_SLIGHT_FILE)
...
did the trick
$ make up
...wait a while...
$ make test
curl localhost:8081/spin/hello
Hello world from Spin!curl localhost:8081/slight/hello
hello world!
meaning, that I am now unblocked for the moment and can try my own workloads with that configuration
Now I'll do the same and try it out....
I will push a write-up on properly setting up docker engine for running wasm/wasi images. Thanks @rumpl !
Run the daemon with sudo ./dockerd -D -H unix:///tmp/docker.sock --data-root /tmp/dockerd --pidfile /tmp/docker.pid
Create a context that will connect to this daemon: docker context create test --docker "host=unix:///tmp/test.sock"
Should the docker context be created to "host=unix:///tmp/docker.sock" instead of "unix:///tmp/test.sock"?