Error "no matching manifest for linux/arm64/v8"
parthpat12 opened this issue · 4 comments
Was able to get Ollama and webui running. But when I try to install "harbor up llamacpp litellm vllm", I get error message.
harbor up works:
Mac-Studio ~ % harbor up [+] Running 21/21 ✔ webui Pulled 59.6s ✔ 92c3b3500be6 Pull complete 2.6s ✔ 89790d4ca55c Pull complete 2.7s ✔ 01ac286af1cc Pull complete 3.3s ✔ 4ba8b3df285e Pull complete 3.3s ✔ 39530234331f Pull complete 3.3s ✔ 4f4fb700ef54 Pull complete 3.3s ✔ 7ea6b4f15cc2 Pull complete 3.3s ✔ cd34c4ad28ac Pull complete 3.3s ✔ eb06520c5bae Pull complete 25.4s ✔ e8d4b7d1f622 Pull complete 25.4s ✔ 1391d428a531 Pull complete 58.1s ✔ 695e24f3f07c Pull complete 58.8s ✔ a0a6c53e8519 Pull complete 58.8s ✔ b27529f13b89 Pull complete 58.8s ✔ 7cc46888e662 Pull complete 59.1s ✔ ollama Pulled 48.3s ✔ e63ce922f022 Pull complete 5.3s ✔ f8b8e7f3f328 Pull complete 25.7s ✔ f6ad96f729db Pull complete 25.8s ✔ 744f4f23d145 Pull complete 47.8s [+] Running 3/3 ✔ Network harbor_harbor-network Creat... 0.0s ✔ Container harbor.ollama Healthy 0.7s ✔ Container harbor.webui Healthy 10.7s
harbor up llamacpp litellm vllm fails:
Mac-Studio ~ % harbor up llamacpp litellm vllm [+] Running 1/3 ⠼ llamacpp Pulling 0.4s ⠼ litellm Pulling 0.4s ✘ litellmdb Error context canceled 0.4s no matching manifest for linux/arm64/v8 in the manifest list entries
Trying individually: harbor up llamacpp also fails
Mac-Studio ~ % harbor up llamacpp [+] Running 0/1 ⠼ llamacpp Pulling 0.4s no matching manifest for linux/arm64/v8 in the manifest list entries
Trying individually: harbor up vllm also fails after package download
Mac-Studio ~ % harbor up vllm [+] Building 147.0s (8/8) FINISHED docker:desktop-linux => [vllm internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 204B 0.0s => [vllm internal] load metadata for docker.io/vllm/vllm-openai:v0.6.0 5.8s => [vllm auth] vllm/vllm-openai:pull token for registry-1.docker.io 0.0s => [vllm internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [vllm 1/2] FROM docker.io/vllm/vllm-openai:v0.6.0@sha256:072427aa6f 133.8s => => resolve docker.io/vllm/vllm-openai:v0.6.0@sha256:072427aa6f95c7478 0.0s => => sha256:43cfb69dbb464ebad014cd4687bf02ee4f5011d54 27.51MB / 27.51MB 0.7s => => sha256:072427aa6f95c74782a9bc3fe1d1fcd1e1aa3fe47b3 2.84kB / 2.84kB 0.0s => => sha256:fbcd35dc5bc3a7bda41926aadd083020f942b001eba 7.94MB / 7.94MB 1.7s => => sha256:c7232af9ae05f7de83f8d6171bd0c35a4dd0a85eb 57.59MB / 57.59MB 1.8s => => sha256:714424fc682cec74d71394991384f2ebd3897b390 11.99kB / 11.99kB 0.0s => => sha256:db6cdef1932a0d9ca6ef9a539e08d491f66d1b1ed81926a 185B / 185B 0.7s => => extracting sha256:43cfb69dbb464ebad014cd4687bf02ee4f5011d540916c65 0.7s => => sha256:56dc8550293751a1604e97ac949cfae82ba20cb2a28 6.89kB / 6.89kB 0.8s => => sha256:9f61b3db38d69dddb7c6f05c299c875b3c7b6d97869e2f9 104B / 104B 0.9s => => sha256:0bd39d0469a8457b392113410f3125a0859d6e5f6d48ad8 230B / 230B 1.0s => => sha256:d22ff5f4aac61b315f6054a8a18ecbe9dcd3af 231.18MB / 231.18MB 10.4s => => extracting sha256:fbcd35dc5bc3a7bda41926aadd083020f942b001ebac6f1d 0.2s => => sha256:d866993704f54dbc8fb066ffc01f4e0e0645617600b 6.90kB / 6.90kB 1.8s => => sha256:b4918d864665c1c0b4ab2df2c6951e0b7ec94e90b4 3.54GB / 3.54GB 84.0s => => sha256:e93cc01aab8b268f0f4cc7c62ca1401d317824b9c7 1.34GB / 1.34GB 41.5s => => extracting sha256:c7232af9ae05f7de83f8d6171bd0c35a4dd0a85ebafb15b9 0.7s => => extracting sha256:db6cdef1932a0d9ca6ef9a539e08d491f66d1b1ed81926ae 0.0s => => extracting sha256:56dc8550293751a1604e97ac949cfae82ba20cb2a28e0347 0.0s => => extracting sha256:9f61b3db38d69dddb7c6f05c299c875b3c7b6d97869e2f96 0.0s => => extracting sha256:0bd39d0469a8457b392113410f3125a0859d6e5f6d48ad85 0.0s => => sha256:fd30840e514de4a28db901cf841543e45fc26221 15.12MB / 15.12MB 11.3s => => extracting sha256:d22ff5f4aac61b315f6054a8a18ecbe9dcd3afad842ea601 5.8s => => extracting sha256:d866993704f54dbc8fb066ffc01f4e0e0645617600bba305 0.0s => => extracting sha256:b4918d864665c1c0b4ab2df2c6951e0b7ec94e90b425d15 37.9s => => extracting sha256:e93cc01aab8b268f0f4cc7c62ca1401d317824b9c76b99c 10.2s => => extracting sha256:fd30840e514de4a28db901cf841543e45fc262213c5eb1de 1.2s => [vllm 2/2] RUN pip install bitsandbytes 6.6s => [vllm] exporting to image 0.7s => => exporting layers 0.7s => => writing image sha256:40b832b504decfe262b9569b4f5cabd345c0693005859 0.0s => => naming to docker.io/library/harbor-vllm 0.0s => [vllm] resolving provenance for metadata file 0.0s [+] Running 3/0 ✔ Container harbor.vllm Created0.1s ✔ Container harbor.ollama Running0.0s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Created0.1s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Running0.0s [+] Running 2/4rbor.webui ⠙ Container harbor.vllm Starting0.3s llm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Running0.0s [+] Running 1/4rbor.webui ⠹ Container harbor.vllm Waiting0.4s vllm The requested image's platform (linux/amd64) does not match the detected ⠙ Container harbor.ollama Waiting0.4s [+] Running 1/4rbor.webui ⠸ Container harbor.vllm Waiting0.5s vllm The requested image's platform (linux/amd64) does not match the detected ⠹ Container harbor.ollama Waiting0.5s [+] Running 1/4rbor.webui ⠼ Container harbor.vllm Waiting0.6s vllm The requested image's platform (linux/amd64) does not match the detected ⠸ Container harbor.ollama Waiting0.6s [+] Running 1/4rbor.webui ⠴ Container harbor.vllm Waiting0.7s vllm The requested image's platform (linux/amd64) does not match the detected ⠼ Container harbor.ollama Waiting0.7s [+] Running 1/4rbor.webui ⠦ Container harbor.vllm Waiting0.8s vllm The requested image's platform (linux/amd64) does not match the detected ⠴ Container harbor.ollama Waiting0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 3/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s [+] Running 4/4rbor.webui ✔ Container harbor.vllm Healthy0.8s vllm The requested image's platform (linux/amd64) does not match the detected ✔ Container harbor.ollama Healthy0.8s ✔ Container harbor.webui Healthy10.8s ! vllm The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 0.0s
The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Are you running on Raspberry PI? Not all services will have docker images with ARM arch, unfortunately, unlike MacOS there's also no x86 translation layer, so they're pretty much incompatible, unfortunately.
Ollama works because maintainers there publish both arm64 and amd64 builds:
https://hub.docker.com/r/ollama/ollama/tags
Neither llama.cpp or vllm have ARM builds, unfortunately:
https://hub.docker.com/r/vllm/vllm-openai/tags
The only workaround as for the moment, is to build an image and then swap it with a pre-built one (by editing the compose.llamacpp.yml). I think llama.cpp should have ARM bins pre-built somewhere, but it's unlikely that vLLM will have them.
P.S. When pasting large pieces of text on GitHub, the best way is to hide it behind the details tag, you can quickly do that by typing "/" on a new line, finding details and hitting "Enter". For code blocks, a triple ticks syntax would give best results:
`single ticks are not very suitable for large chunks of code`
```python
# Triple ticks are the best and will even give syntax highlight
\```
I've mentioned ARM builds in the Requirements section in the README
I maybe missing something but according to llama.cpp GH page:
Apple silicon is a first-class citizen - optimized via ARM NEON, Accelerate and Metal frameworks
Is the issue occurring because it's running from docker?
Yes, that's an Apple and Docker thing, llama.cpp itself can be compiled and run on MacOS, however Apple doesn't support GPU passthrough in their hypervisor, so any kind of GPU optimisation targeting Apple silicon wouldn't work in the container. CPU-only inference would still work quite well compared to non-apple hardware, which is exactly how ollama worked in your initial setup above. The situation was like this for a long while, so it's unlikely that Apple will change things in this aspect.
On top of that, Docker images still must have versions for ARM specifically, as for now. Unfortunately, llama.cpp do not publish such ARM images, hence the situation you observed