wasmCloud/examples

Installing via Docker Compose Fails

Closed this issue · 7 comments

snnsnn commented

Hi, I follow the instruction from https://wasmcloud.dev/overview/installation/install-with-docker/ but failed to run a working setup.

Steps to recreate:

  1. Cd into docker/ directory
  2. Run docker compose up command.

Docker version: Docker version 20.10.18, build b40c2f6 (Currently the latest build)
OS: Ubuntu 22.04.1 LTS

Restarting container also returns "OTLP exporter failed to initialize with exception :throw:{:application_either_not_started_or_not_ready, :tls_certificate_check}" error.

Here is the log from the wasmcloud container:

docker logs -f docker-wasmcloud-1 
19:07:48.824 [warning] OTLP exporter failed to initialize with exception :throw:{:application_either_not_started_or_not_ready, :tls_certificate_check}
19:10:00.763 [error] Failed to configure invocation chunking object store (NATS): "Os { code: 110, kind: TimedOut, message: \"Connection timed out\" }". Any chunked invocations will fail.
19:10:00.763 [info] OTLP exporter successfully initialized
19:10:00.764 [info] Wrote configuration file host_config.json
19:10:00.764 [info] Wrote configuration file /root/.wash/host_config.json
19:10:00.764 [info] Connecting to control interface NATS without authentication
19:10:00.765 [info] Connecting to lattice rpc NATS without authentication
19:10:00.767 [info] Host NBJPRKFYNT3PMBIUQKL6KSUTPUDUDFXAVPBNPSLEPJNCN2QM32DS6Z67 (summer-glade-5233) started.
19:10:00.767 [info] Host issuer public key: CCOVHI4XN3QSZAT5MLXXSFMS5MHIWH7LOSTF5OX2JRYH7ECNGYOF4YOJ
19:10:00.767 [info] Valid cluster signers: CCOVHI4XN3QSZAT5MLXXSFMS5MHIWH7LOSTF5OX2JRYH7ECNGYOF4YOJ
19:10:00.767 [warning] WARNING. You are using an ad hoc generated cluster seed.
For any other host or CLI tool to communicate with this host,
you MUST copy the following seed key and use it as the value
of the WASMCLOUD_CLUSTER_SEED environment variable:

SCAACHUDH7UWPPRLZMEZKC3TJMKAEUSAYK4AEF54OU7PQIUGVCBEOK5N5U

You must also ensure the following cluster signer is in the list of valid
signers for any new host you start:

CCOVHI4XN3QSZAT5MLXXSFMS5MHIWH7LOSTF5OX2JRYH7ECNGYOF4YOJ


19:10:03.767 [error] failed to connect :timeout
19:10:03.767 [error] failed to connect :timeout
19:10:03.767 [error] failed to connect :timeout
19:10:03.767 [error] failed to connect :timeout
19:10:03.768 [error] Elixir.Gnat.ConsumerSupervisor unexpected shutdown {:timeout, {GenServer, :call, [#PID<0.2055.0>, {:sub, #PID<0.2061.0>, "_INBOX.6YNvN3r", []}, 5000]}}
19:10:03.768 [error] Elixir.Gnat.ConsumerSupervisor unexpected shutdown {:timeout, {GenServer, :call, [#PID<0.2055.0>, {:sub, #PID<0.2059.0>, "wasmbus.ctl.default.registries.put", []}, 5000]}}
19:10:03.768 [error] GenServer #PID<0.2059.0> terminating
** (stop) exited in: GenServer.call(#PID<0.2055.0>, {:sub, #PID<0.2059.0>, "wasmbus.ctl.default.registries.put", []}, 5000)
    ** (EXIT) time out
    (elixir 1.13.3) lib/gen_server.ex:1030: GenServer.call/3
    (gnat 1.5.2) lib/gnat.ex:168: Gnat.sub/4
    (gnat 1.5.2) lib/gnat/consumer_supervisor.ex:81: anonymous fn/2 in Gnat.ConsumerSupervisor.handle_info/2
    (elixir 1.13.3) lib/enum.ex:1593: Enum."-map/2-lists^map/1-0-"/2
    (gnat 1.5.2) lib/gnat/consumer_supervisor.ex:78: Gnat.ConsumerSupervisor.handle_info/2
    (stdlib 3.17.1) gen_server.erl:695: :gen_server.try_dispatch/4
    (stdlib 3.17.1) gen_server.erl:771: :gen_server.handle_msg/6
    (stdlib 3.17.1) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
Last message: :connect
19:10:03.768 [error] GenServer #PID<0.2061.0> terminating
** (stop) exited in: GenServer.call(#PID<0.2055.0>, {:sub, #PID<0.2061.0>, "_INBOX.6YNvN3r", []}, 5000)
    ** (EXIT) time out
    (elixir 1.13.3) lib/gen_server.ex:1030: GenServer.call/3
    (gnat 1.5.2) lib/gnat.ex:168: Gnat.sub/4
    (gnat 1.5.2) lib/gnat/consumer_supervisor.ex:81: anonymous fn/2 in Gnat.ConsumerSupervisor.handle_info/2
    (elixir 1.13.3) lib/enum.ex:1593: Enum."-map/2-lists^map/1-0-"/2
    (gnat 1.5.2) lib/gnat/consumer_supervisor.ex:78: Gnat.ConsumerSupervisor.handle_info/2
    (stdlib 3.17.1) gen_server.erl:695: :gen_server.try_dispatch/4
    (stdlib 3.17.1) gen_server.erl:771: :gen_server.handle_msg/6
    (stdlib 3.17.1) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
Last message: :connect
19:10:03.769 [error] GenServer #PID<0.2060.0> terminating
** (stop) exited in: GenServer.call(#PID<0.2055.0>, {:sub, #PID<0.2059.0>, "wasmbus.ctl.default.registries.put", []}, 5000)
    ** (EXIT) time out
Last message: {:EXIT, #PID<0.2059.0>, {:timeout, {GenServer, :call, [#PID<0.2055.0>, {:sub, #PID<0.2059.0>, "wasmbus.ctl.default.registries.put", []}, 5000]}}}
19:10:03.769 [error] GenServer #PID<0.2062.0> terminating
** (stop) exited in: GenServer.call(#PID<0.2055.0>, {:sub, #PID<0.2061.0>, "_INBOX.6YNvN3r", []}, 5000)
    ** (EXIT) time out
Last message: {:EXIT, #PID<0.2061.0>, {:timeout, {GenServer, :call, [#PID<0.2055.0>, {:sub, #PID<0.2061.0>, "_INBOX.6YNvN3r", []}, 5000]}}}
19:10:03.770 [info] Started wasmCloud OTP Host Runtime
19:10:03.770 [notice] Application host_core exited: HostCore.start(:normal, []) returned an error: shutdown: failed to start child: HostCore.Host
    ** (EXIT) exited in: GenServer.call(:control_nats, {:pub, "wasmbus.evt.default", "{\"data\":{\"friendly_name\":\"summer-glade-5233\",\"labels\":{\"hostcore.arch\":\"x86_64\",\"hostcore.os\":\"linux\",\"hostcore.osfamily\":\"unix\"}},\"datacontenttype\":\"application/json\",\"id\":\"cc73c69f-84d0-4c10-9058-51d8c41602ec\",\"source\":\"NBJPRKFYNT3PMBIUQKL6KSUTPUDUDFXAVPBNPSLEPJNCN2QM32DS6Z67\",\"specversion\":\"1.0\",\"time\":\"2022-10-03T19:10:00.767560Z\",\"type\":\"com.wasmcloud.lattice.host_started\"}", [headers: []]}, 5000)
        ** (EXIT) time out
{"Kernel pid terminated",application_controller,"{application_start_failure,host_core,{{shutdown,{failed_to_start_child,'Elixir.HostCore.Host',{timeout,{'Elixir.GenServer',call,[control_nats,{pub,<<\"wasmbus.evt.default\">>,<<\"{\\"data\\":{\\"friendly_name\\":\\"summer-glade-5233\\",\\"labels\\":{\\"hostcore.arch\\":\\"x86_64\\",\\"hostcore.os\\":\\"linux\\",\\"hostcore.osfamily\\":\\"unix\\"}},\\"datacontenttype\\":\\"application/json\\",\\"id\\":\\"cc73c69f-84d0-4c10-9058-51d8c41602ec\\",\\"source\\":\\"NBJPRKFYNT3PMBIUQKL6KSUTPUDUDFXAVPBNPSLEPJNCN2QM32DS6Z67\\",\\"specversion\\":\\"1.0\\",\\"time\\":\\"2022-10-03T19:10:00.767560Z\\",\\"type\\":\\"com.wasmcloud.lattice.host_started\\"}\">>,[{headers,[]}]},5000]}}}},{'Elixir.HostCore',start,[normal,[]]}}}"}
Kernel pid terminated (application_controller) ({application_start_failure,host_core,{{shutdown,{failed_to_start_child,'Elixir.HostCore.Host',{timeout,{'Elixir.GenServer',call,[control_nats,{pub,<<"wasmbus.evt.default">>,<<"{\"data\":{\"friendly_name\":\"summer-glade-5233\",\"labels\":{\"hostcore.arch\":\"x86_64\",\"hostcore.os\":\"linux\",\"hostcore.osfamily\":\"unix\"}},\"datacontenttype\":\"application/json\",\"id\":\"cc73c69f-84d0-4c10-9058-51d8c41602ec\",\"source\":\"NBJPRKFYNT3PMBIUQKL6KSUTPUDUDFXAVPBNPSLEPJNCN2QM32DS6Z67\",\"specversion\":\"1.0\",\"time\":\"2022-10-03T19:10:00.767560Z\",\"type\":\"com.wasmcloud.lattice.host_started\"}">>,[{headers,[]}]},5000]}}}},{'Elixir.HostCore',start,[normal,[]]}}})

Crash dump is being written to: erl_crash.dump...19:11:43.939 [warning] OTLP exporter failed to initialize with exception :throw:{:application_either_not_started_or_not_ready, :tls_certificate_check}
19:13:54.235 [error] Failed to configure invocation chunking object store (NATS): "Os { code: 110, kind: TimedOut, message: \"Connection timed out\" }". Any chunked invocations will fail.
19:13:54.235 [info] OTLP exporter successfully initialized
19:13:54.235 [info] Wrote configuration file host_config.json
19:13:54.236 [info] Wrote configuration file /root/.wash/host_config.json
19:13:54.236 [info] Connecting to control interface NATS without authentication
19:13:54.236 [info] Connecting to lattice rpc NATS without authentication
19:13:54.239 [info] Host NC7CVRVBJIOUWBWBVSRBTACVNWEZBC4NIJCBFWPVZQ6INAHS3FALJAO6 (twilight-haze-8374) started.
19:13:54.239 [info] Host issuer public key: CCOVHI4XN3QSZAT5MLXXSFMS5MHIWH7LOSTF5OX2JRYH7ECNGYOF4YOJ
19:13:54.239 [info] Valid cluster signers: CCOVHI4XN3QSZAT5MLXXSFMS5MHIWH7LOSTF5OX2JRYH7ECNGYOF4YOJ
19:13:57.239 [error] failed to connect :timeout
19:13:57.239 [error] failed to connect :timeout
19:13:57.239 [error] failed to connect :timeout
19:13:57.239 [error] failed to connect :timeout
19:13:57.239 [error] Elixir.Gnat.ConsumerSupervisor unexpected shutdown {:timeout, {GenServer, :call, [#PID<0.2052.0>, {:sub, #PID<0.2058.0>, "wasmbus.ctl.default.registries.put", []}, 5000]}}
19:13:57.239 [error] Elixir.Gnat.ConsumerSupervisor unexpected shutdown {:timeout, {GenServer, :call, [#PID<0.2052.0>, {:sub, #PID<0.2060.0>, "_INBOX.8jO7Rba", []}, 5000]}}
19:13:57.239 [error] GenServer #PID<0.2060.0> terminating
** (stop) exited in: GenServer.call(#PID<0.2052.0>, {:sub, #PID<0.2060.0>, "_INBOX.8jO7Rba", []}, 5000)
    ** (EXIT) time out
    (elixir 1.13.3) lib/gen_server.ex:1030: GenServer.call/3
    (gnat 1.5.2) lib/gnat.ex:168: Gnat.sub/4
    (gnat 1.5.2) lib/gnat/consumer_supervisor.ex:81: anonymous fn/2 in Gnat.ConsumerSupervisor.handle_info/2
    (elixir 1.13.3) lib/enum.ex:1593: Enum."-map/2-lists^map/1-0-"/2
    (gnat 1.5.2) lib/gnat/consumer_supervisor.ex:78: Gnat.ConsumerSupervisor.handle_info/2
    (stdlib 3.17.1) gen_server.erl:695: :gen_server.try_dispatch/4
    (stdlib 3.17.1) gen_server.erl:771: :gen_server.handle_msg/6
    (stdlib 3.17.1) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
Last message: :connect
19:13:57.239 [error] GenServer #PID<0.2058.0> terminating
** (stop) exited in: GenServer.call(#PID<0.2052.0>, {:sub, #PID<0.2058.0>, "wasmbus.ctl.default.registries.put", []}, 5000)
    ** (EXIT) time out
    (elixir 1.13.3) lib/gen_server.ex:1030: GenServer.call/3
    (gnat 1.5.2) lib/gnat.ex:168: Gnat.sub/4
    (gnat 1.5.2) lib/gnat/consumer_supervisor.ex:81: anonymous fn/2 in Gnat.ConsumerSupervisor.handle_info/2
    (elixir 1.13.3) lib/enum.ex:1593: Enum."-map/2-lists^map/1-0-"/2
    (gnat 1.5.2) lib/gnat/consumer_supervisor.ex:78: Gnat.ConsumerSupervisor.handle_info/2
    (stdlib 3.17.1) gen_server.erl:695: :gen_server.try_dispatch/4
    (stdlib 3.17.1) gen_server.erl:771: :gen_server.handle_msg/6
    (stdlib 3.17.1) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
Last message: :connect
19:13:57.240 [error] GenServer #PID<0.2061.0> terminating
** (stop) exited in: GenServer.call(#PID<0.2052.0>, {:sub, #PID<0.2060.0>, "_INBOX.8jO7Rba", []}, 5000)
    ** (EXIT) time out
Last message: {:EXIT, #PID<0.2060.0>, {:timeout, {GenServer, :call, [#PID<0.2052.0>, {:sub, #PID<0.2060.0>, "_INBOX.8jO7Rba", []}, 5000]}}}
19:13:57.240 [error] GenServer #PID<0.2059.0> terminating
** (stop) exited in: GenServer.call(#PID<0.2052.0>, {:sub, #PID<0.2058.0>, "wasmbus.ctl.default.registries.put", []}, 5000)
    ** (EXIT) time out
Last message: {:EXIT, #PID<0.2058.0>, {:timeout, {GenServer, :call, [#PID<0.2052.0>, {:sub, #PID<0.2058.0>, "wasmbus.ctl.default.registries.put", []}, 5000]}}}
19:13:57.242 [info] Started wasmCloud OTP Host Runtime
19:13:57.242 [notice] Application host_core exited: HostCore.start(:normal, []) returned an error: shutdown: failed to start child: HostCore.Host
    ** (EXIT) exited in: GenServer.call(:control_nats, {:pub, "wasmbus.evt.default", "{\"data\":{\"friendly_name\":\"twilight-haze-8374\",\"labels\":{\"hostcore.arch\":\"x86_64\",\"hostcore.os\":\"linux\",\"hostcore.osfamily\":\"unix\"}},\"datacontenttype\":\"application/json\",\"id\":\"8746a989-14ed-418f-aebe-e02bc5280b98\",\"source\":\"NC7CVRVBJIOUWBWBVSRBTACVNWEZBC4NIJCBFWPVZQ6INAHS3FALJAO6\",\"specversion\":\"1.0\",\"time\":\"2022-10-03T19:13:54.239396Z\",\"type\":\"com.wasmcloud.lattice.host_started\"}", [headers: []]}, 5000)
        ** (EXIT) time out
{"Kernel pid terminated",application_controller,"{application_start_failure,host_core,{{shutdown,{failed_to_start_child,'Elixir.HostCore.Host',{timeout,{'Elixir.GenServer',call,[control_nats,{pub,<<\"wasmbus.evt.default\">>,<<\"{\\"data\\":{\\"friendly_name\\":\\"twilight-haze-8374\\",\\"labels\\":{\\"hostcore.arch\\":\\"x86_64\\",\\"hostcore.os\\":\\"linux\\",\\"hostcore.osfamily\\":\\"unix\\"}},\\"datacontenttype\\":\\"application/json\\",\\"id\\":\\"8746a989-14ed-418f-aebe-e02bc5280b98\\",\\"source\\":\\"NC7CVRVBJIOUWBWBVSRBTACVNWEZBC4NIJCBFWPVZQ6INAHS3FALJAO6\\",\\"specversion\\":\\"1.0\\",\\"time\\":\\"2022-10-03T19:13:54.239396Z\\",\\"type\\":\\"com.wasmcloud.lattice.host_started\\"}\">>,[{headers,[]}]},5000]}}}},{'Elixir.HostCore',start,[normal,[]]}}}"}
Kernel pid terminated (application_controller) ({application_start_failure,host_core,{{shutdown,{failed_to_start_child,'Elixir.HostCore.Host',{timeout,{'Elixir.GenServer',call,[control_nats,{pub,<<"wasmbus.evt.default">>,<<"{\"data\":{\"friendly_name\":\"twilight-haze-8374\",\"labels\":{\"hostcore.arch\":\"x86_64\",\"hostcore.os\":\"linux\",\"hostcore.osfamily\":\"unix\"}},\"datacontenttype\":\"application/json\",\"id\":\"8746a989-14ed-418f-aebe-e02bc5280b98\",\"source\":\"NC7CVRVBJIOUWBWBVSRBTACVNWEZBC4NIJCBFWPVZQ6INAHS3FALJAO6\",\"specversion\":\"1.0\",\"time\":\"2022-10-03T19:13:54.239396Z\",\"type\":\"com.wasmcloud.lattice.host_started\"}">>,[{headers,[]}]},5000]}}}},{'Elixir.HostCore',start,[normal,[]]}}})

Crash dump is being written to: erl_crash.dump...done
ricochet commented

I couldn't rep, but I have a different setup (macos m1 and docker desktop).

I checked the wasmcloud host version is the latest.

docker compose exec wasmcloud bash
./bin/wasmcloud_host version
wasmcloud_host 0.57.4

Extra info on the docker and docker compose versions that I'm running with:

docker compose version
Docker Compose version v2.10.2

docker version
Client:
 Cloud integration: v1.0.29
 Version:           20.10.17
 API version:       1.41
 Go version:        go1.17.11
 Git commit:        100c701
 Built:             Mon Jun  6 23:04:45 2022
 OS/Arch:           darwin/arm64
 Context:           default
 Experimental:      true

Server: Docker Desktop 4.12.0 (85629)
 Engine:
  Version:          20.10.17
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.17.11
  Git commit:       a89b842
  Built:            Mon Jun  6 23:01:01 2022
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.8
  GitCommit:        9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
brooksmtownsend commented

@snnsnn One thing I would check for is just that you don't have a very old version of the wasmCloud container, wasmcloud/wasmcloud_host:latest in your docker image list. Also, this docker compose should specify that the wasmCloud container needs to start after nats, redis, grafana, and tempo, perhaps your error is what happens when wasmCloud starts before all of those services.

I can reproduce the OTLP error if I remove all of those containers

20:28:16.460 [warning] OTLP exporter failed to initialize with exception :throw:{:application_either_not_started_or_not_ready, :tls_certificate_check}

However it seems that docker compose is smart enough with my setup to download all the containers first. I'd bet this issue can be solved by just enforcing an order for container startup

brooksmtownsend commented

@snnsnn try #167 and see if this fixes your issue 😄

snnsnn commented

I used the file from #167 still could not get it working 🙈. I attached the logs.

Here is the content of the docker compose file.

# This docker-compose file loads 
#   nats with JetStream enabled
#   a local OCI registry
#   redis (for the kvredis capability provider)
#   grafana + tempo for tracing
#   wasmcloud_host

version: "4"
services:
  nats:
    image: nats:2.8.4-alpine
    ports:
      - "4222:4222"
      - "6222:6222"
      - "8222:8222"
    command: [ "-js" ]
  redis:
    image: redis:6.2
    ports:
      - "6379:6379"
  registry:
    image: registry:2.7
    ports:
      - "5000:5000"
  grafana:
    image: grafana/grafana:8.1.2
    ports:
      - 5050:3000
    volumes:
      - ./grafana-datasources.yaml:/etc/grafana/provisioning/datasources/datasources.yaml
    environment:
      - GF_AUTH_ANONYMOUS_ENABLED=true
      - GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
      - GF_AUTH_DISABLE_LOGIN_FORM=true
    depends_on:
      - tempo  
  tempo:
    image: grafana/tempo:1.1.0
    command: ["-config.file=/etc/tempo.yaml"]
    volumes:
      - ./tempo.yaml:/etc/tempo.yaml
    ports:
      - 8000:8000 # tempo
      - 55681:55681 # otlp http  

  wasmcloud:
    depends_on:
      - "nats"
      - "redis"
      - "grafana"
      - "tempo"
    image: wasmcloud/wasmcloud_host:latest
    environment:
      RUST_LOG: debug,hyper=info
      WASMCLOUD_RPC_HOST: nats
      WASMCLOUD_CTL_HOST: nats
      WASMCLOUD_PROV_RPC_HOST: nats
      OTEL_TRACES_EXPORTER: otlp
      OTEL_EXPORTER_OTLP_ENDPOINT: http://tempo:55681
    ports:
      - "4000:4000"
      - "8080-8089:8080-8089" # Allows exposing examples on ports 8080-8089

  wadm:
    depends_on:
      - "nats"
      - "redis"
    image: wasmcloud.azurecr.io/wadm:0.1.0
    environment:
      - WADM_NATS_HOST=nats
      - WADM_REDIS_HOST=redis
docker compose version
Docker Compose version v2.10.2

docker version
Client: Docker Engine - Community
 Version:           20.10.18
 API version:       1.41
 Go version:        go1.18.6
 Git commit:        b40c2f6
 Built:             Thu Sep  8 23:11:43 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.18
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.6
  Git commit:       e42327a
  Built:            Thu Sep  8 23:09:30 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.8
  GitCommit:        9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Cleaned all containers beforehand, no running or failed container, then run following command.

 docker compose -f updated-compose.yml up > logs.txt 2>&1

Here is logs.txt

Running on custom desktop with 16 core and 64gb ram. Ubuntu 22.04.1 LTS

docker ps -a
CONTAINER ID   IMAGE                             COMMAND                  CREATED          STATUS                       PORTS                                                                                                                             NAMES
773c029f80e3   wasmcloud.azurecr.io/wadm:0.1.0   "/opt/app/bin/wadm s…"   10 minutes ago   Up 6 minutes                                                                                                                                                   docker-wadm-1
f7e67cd02fca   wasmcloud/wasmcloud_host:latest   "/opt/app/bin/wasmcl…"   23 minutes ago   Exited (139) 4 minutes ago                                                                                                                                     docker-wasmcloud-1
131f2f0e49be   grafana/grafana:8.1.2             "/run.sh"                23 minutes ago   Up 6 minutes                 0.0.0.0:5050->3000/tcp, :::5050->3000/tcp                                                                                         docker-grafana-1
2c7ce238f86a   registry:2.7                      "/entrypoint.sh /etc…"   23 minutes ago   Up 6 minutes                 0.0.0.0:5000->5000/tcp, :::5000->5000/tcp                                                                                         docker-registry-1
ee3f962410d1   redis:6.2                         "docker-entrypoint.s…"   23 minutes ago   Up 6 minutes                 0.0.0.0:6379->6379/tcp, :::6379->6379/tcp                                                                                         docker-redis-1
e2a06d825773   nats:2.8.4-alpine                 "docker-entrypoint.s…"   23 minutes ago   Up 6 minutes                 0.0.0.0:4222->4222/tcp, :::4222->4222/tcp, 0.0.0.0:6222->6222/tcp, :::6222->6222/tcp, 0.0.0.0:8222->8222/tcp, :::8222->8222/tcp   docker-nats-1
d8a208dcbd97   grafana/tempo:1.1.0               "/tempo -config.file…"   23 minutes ago   Up 6 minutes                 0.0.0.0:8000->8000/tcp, :::8000->8000/tcp, 0.0.0.0:55681->55681/tcp, :::55681->55681/tcp                                          docker-tempo-1

I restarted wasmcloud container but failed again:

docker ps -a
CONTAINER ID   IMAGE                             COMMAND                  CREATED          STATUS                          PORTS                                                                                                                             NAMES
773c029f80e3   wasmcloud.azurecr.io/wadm:0.1.0   "/opt/app/bin/wadm s…"   14 minutes ago   Up 11 minutes                                                                                                                                                     docker-wadm-1
f7e67cd02fca   wasmcloud/wasmcloud_host:latest   "/opt/app/bin/wasmcl…"   27 minutes ago   Exited (1) About a minute ago                                                                                                                                     docker-wasmcloud-1
131f2f0e49be   grafana/grafana:8.1.2             "/run.sh"                27 minutes ago   Up 11 minutes                   0.0.0.0:5050->3000/tcp, :::5050->3000/tcp                                                                                         docker-grafana-1
2c7ce238f86a   registry:2.7                      "/entrypoint.sh /etc…"   27 minutes ago   Up 11 minutes                   0.0.0.0:5000->5000/tcp, :::5000->5000/tcp                                                                                         docker-registry-1
ee3f962410d1   redis:6.2                         "docker-entrypoint.s…"   27 minutes ago   Up 11 minutes                   0.0.0.0:6379->6379/tcp, :::6379->6379/tcp                                                                                         docker-redis-1
e2a06d825773   nats:2.8.4-alpine                 "docker-entrypoint.s…"   27 minutes ago   Up 11 minutes                   0.0.0.0:4222->4222/tcp, :::4222->4222/tcp, 0.0.0.0:6222->6222/tcp, :::6222->6222/tcp, 0.0.0.0:8222->8222/tcp, :::8222->8222/tcp   docker-nats-1
d8a208dcbd97   grafana/tempo:1.1.0               "/tempo -config.file…"   27 minutes ago   Up 11 minutes                   0.0.0.0:8000->8000/tcp, :::8000->8000/tcp, 0.0.0.0:55681->55681/tcp, :::55681->55681/tcp

I checked the host version while container is running:

docker compose exec wasmcloud bash ./bin/wasmcloud_host version
wasmcloud_host 0.57.4
brooksmtownsend commented

@snnsnn it certainly seems like wadm and wasmcloud are both having trouble connecting to NATS. I'm not an expert with Docker networking, but the compose file should create a network so that the containers can resolve each other by their name (e.g. nats) so I'm surprised it doesn't work out of the box.

Could you try one more thing for me, docker exec into the wasmcloud_host container, use apt to apt install netcat and see if you can run nc 127.0.0.1 4222 -vz and nc nats 4222 -vz to see if you can successfully connect? If you run docker network ls it should also show you created networks. Thank you in advance

snnsnn commented

As you asked, I bash into the container docker exec -it docker-wasmcloud-1 bash and installed netcat.

if you can run nc 127.0.0.1 4222 -vz

nc 127.0.0.1 4222 -vz
nc: connect to 127.0.0.1 port 4222 (tcp) failed: Connection refused

and nc nats 4222 -vz

This one gets stuck and returns no response until container exits and session ends.

docker network ls
NETWORK ID     NAME             DRIVER    SCOPE
921c20fc9250   bridge           bridge    local
c06a0e2f9ebf   docker_default   bridge    local
5fb5115f5b10   host             host      local
8869ef663da8   none             null      local

I also tried it to run on another Ubuntu 22.04.1 LTS with Docker version 20.10.18, got the same results.

snnsnn commented

Yes, it was network error, containers did not have access to services. I reinstalled docker and restarted computer, now everything works as expected. Thank you for your time.

  • 🚀2