Failed to build latest `wasi-sdk` from source

Since wasi-sdk is only released officially for amd64, we need builds for other machine types such as aarch64. Therefore, I have written a minimal Dockerfile that attempts to build wasi-sdk from source:

FROM ubuntu:22.04 as base
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y \
	cmake \
	build-essential \
	git \
	ninja-build \
	python3 \
	wget
RUN adduser wasisdk
USER wasisdk
WORKDIR /home/wasisdk
RUN git clone https://github.com/WebAssembly/wasi-sdk
WORKDIR wasi-sdk
# src/llvm-project is a *huge* project. Pulling all of its history can take
# way too long, and we are not interested in it, after all.
# --force and --remote are required to avoid the following error from git:
# "error: Server does not allow request for unadvertised object"
RUN git submodule update --init --recursive --depth 1 --force --remote
RUN make package
FROM scratch
COPY --from=base /home/wasisdk/wasi-sdk/dist/ /opt/wasi-sdk/

However, after a couple of hours building the project, libc++abi.so fails to build:

 > [base 8/8] RUN make package:
7860.3 [1040/1141] Building CXX object libcxxabi/src/CMakeFiles/cxxabi_shared_objects.dir/cxa_demangle.cpp.o
7860.3 [1041/1141] Linking CXX shared library lib/libc++abi.so
7860.3 FAILED: lib/libc++abi.so 
7860.3 : && /home/wasisdk/wasi-sdk/build/install/opt/wasi-sdk/bin/clang++ --target=wasm32-wasi --sysroot=/home/wasisdk/wasi-sdk/build/install/opt/wasi-sdk/share/wasi-sysroot -fPIC -fdebug-prefix-map=/home/wasisdk/wasi-sdk=wasisdk://v21.4ga1b80857c490+m  --target=wasm32-wasi-preview2 -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections  -Wl,-z,defs -Wl,--color-diagnostics  -nostdlib++ --unwindlib=none -shared  -o lib/libc++abi.so libcxxabi/src/CMakeFiles/cxxabi_shared_objects.dir/cxa_aux_runtime.cpp.o libcxxabi/src/CMakeFiles/cxxabi_shared_objects.dir/cxa_default_handlers.cpp.o libcxxabi/src/CMakeFiles/cxxabi_shared_objects.dir/cxa_demangle.cpp.o libcxxabi/src/CMakeFiles/cxxabi_shared_objects.dir/cxa_exception_storage.cpp.o libcxxabi/src/CMakeFiles/cxxabi_shared_objects.dir/cxa_guard.cpp.o libcxxabi/src/CMakeFiles/cxxabi_shared_objects.dir/cxa_handlers.cpp.o libcxxabi/src/CMakeFiles/cxxabi_shared_objects.dir/cxa_vector.cpp.o libcxxabi/src/CMakeFiles/cxxabi_shared_objects.dir/cxa_virtual.cpp.o libcxxabi/src/CMakeFiles/cxxabi_shared_objects.dir/stdlib_exception.cpp.o libcxxabi/src/CMakeFiles/cxxabi_shared_objects.dir/stdlib_stdexcept.cpp.o libcxxabi/src/CMakeFiles/cxxabi_shared_objects.dir/stdlib_typeinfo.cpp.o libcxxabi/src/CMakeFiles/cxxabi_shared_objects.dir/abort_message.cpp.o libcxxabi/src/CMakeFiles/cxxabi_shared_objects.dir/fallback_malloc.cpp.o libcxxabi/src/CMakeFiles/cxxabi_shared_objects.dir/private_typeinfo.cpp.o libcxxabi/src/CMakeFiles/cxxabi_shared_objects.dir/stdlib_new_delete.cpp.o libcxxabi/src/CMakeFiles/cxxabi_shared_objects.dir/cxa_noexception.cpp.o  -lc  -lunwind_shared  -lc && :
7860.3 wasm-ld: warning: unknown -z value: defs
7860.3 wasm-ld: error: unable to find library -lunwind_shared
7860.3 clang++: error: linker command failed with exit code 1 (use -v to see invocation)
7860.3 [1042/1141] Building CXX object libcxxabi/src/CMakeFiles/cxxabi_static_objects.dir/fallback_malloc.cpp.o
7860.3 [1043/1141] Building CXX object libcxxabi/src/CMakeFiles/cxxabi_static_objects.dir/cxa_demangle.cpp.o
7860.3 ninja: build stopped: subcommand failed.

I am unsure whether the error above is related to the Docker image configuration, such as a missing dependency. Building LLVM from source takes a long while, so unfortunately a trial-and-error process to find out the root issue is very time-costly.

Do you know why this issue occurs, and how it could be solved?

our ci builds and pushes docker images for arm64.

wasi-sdk/.github/workflows/main.yml

Line 188 in a1b8085

platforms: linux/amd64,linux/arm64

i guess you can follow/use it.

RUN git submodule update --init --recursive --depth 1 --force --remote

i suppose this checks out wrong versions.

i suppose this checks out wrong versions.

Could you please explain why? I see the CI workflow uses --depth 32 to avoid the "error: Server does not allow request for unadvertised object" condition, but this could be apparently solved with --force --remote as well. Do you think this is the wrong approach?

i suppose this checks out wrong versions.

Could you please explain why? I see the CI workflow uses --depth 32 to avoid the "error: Server does not allow request for unadvertised object" condition, but this could be apparently solved with --force --remote as well. Do you think this is the wrong approach?

it's just my reading of documentation of --remote in git submodule --help

@midokura-xavi92 do you still think this is a problem in wasi-sdk? if so, can you explain? otherwise, can you close?

It looks like the correct version was chosen with a big enough value for --depth, so I assume this ticket can be closed. Thank you!