Add support for statically-linked musl libc Linux targets (-unknown-linux-musl)

Question

Add support for statically-linked musl libc Linux targets (-unknown-linux-musl)

briansmith opened this issue 6 years ago · 16 comments

Answer 1 · 2019-05-29T11:25:03.000Z

Why is removing the libc crate dependency a goal ?

Answer 2 · 2020-01-11T15:07:30.000Z

FYI - from updates on rust-lang/libc#659, getrandom is now available in libc on most Linux platforms. The holdout is not musl, but android/Bionic. I suggest decoupling the two parts of this issue an moving the getrandom portion into a separate bug.

Answer 3 · 2020-06-11T16:02:23.000Z

More discussion in #935

Answer 4 · 2020-10-28T02:26:12.000Z

@aig787's PR #1067 added the -musl targets for Linux x86_64 and i686 to CI/CD.

Still to do:

rust-lang/rust#70740 indicates that we need to verify that we're doing the right thing w.r.t. PIE (position independent code), especially when compiling the C and assembly code.
Add Aarch64 musl targets to CI/CD.
Minimize the dependencies installed. Currently the i686 job installs "libc6-dev-i386" and "gcc-multilib", but neither of those should be needed for -musl. (No longer relevant.)

Answer 5 · 2020-10-28T05:12:56.000Z

rust-lang/cc-rs#436 indicates that we should be doing more sanity checks to make sure we're not using the wrong header files. e.g. statically asserting that sizeof(uint64_t) == 8.

rust-lang/rust#70740 indicates that we need to verify that we're doing the right thing w.r.t. PIE (position independent code), especially when compiling the C and assembly code.

See also rust-lang/cc-rs#165. We should have a step in CI/CD that verifies that we support PIE correctly.

Answer 6 · 2020-10-28T06:03:45.000Z

Minimize the dependencies installed. Currently the i686 job installs "libc6-dev-i386" and "gcc-multilib", but neither of those should be needed for -musl.

I attempted to remove the gcc-multilib dependency but the build fails with:

 running "clang" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "--target=i686-unknown-linux-musl" "-I" "include" "-Wall" "-Wextra" "-std=c1x" "-Wbad-function-cast" "-Wnested-externs" "-Wstrict-prototypes" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wconversion" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wsign-conversion" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-fstack-protector" "-g3" "-Werror" "-U_FORTIFY_SOURCE" "-c" "-o/home/travis/build/briansmith/ring/target/i686-unknown-linux-musl/debug/build/ring-bb6253d99512d637/out/aes_nohw.o" "crypto/fipsmodule/aes/aes_nohw.c"
  In file included from crypto/fipsmodule/aes/aes_nohw.c:15:
  In file included from include/GFp/aes.h:52:
  In file included from include/GFp/base.h:66:
  In file included from /usr/local/clang-7.0.0/lib/clang/7.0.0/include/stdint.h:63:
  /usr/include/stdint.h:26:10: fatal error: 'bits/libc-header-start.h' file not found
  #include <bits/libc-header-start.h>
           ^~~~~~~~~~~~~~~~~~~~~~~~~~

I doubt using gcc-multilib with musl is the right thing to do but I could be convinced otherwise. I'll try experimenting with using --nostdlibinc and/or -nostdlib but it isn't clear that musl libc supports that configuration either.

Help understanding and/or resolving this is appreciated. Otherwise, maybe this is a good motivation to get rid of all the remaining C code in ring.

Answer 7 · 2020-10-28T09:25:34.000Z

I'll try experimenting with using --nostdlibinc and/or -nostdlib

I did the experiment and that seems like it will work fine. I actually had done the same something similar for wasm32-unknown already and I verified that the same works for musl, without gcc-multilib and libc6-dev-i386. I need to clean up the experiment before creating a PR for it.

Answer 8 · 2020-11-24T07:21:09.000Z

PR #1118 makes progress here; see that PR for details.

Remaining work to be done:

Add static analysis to the build system that verifies that the tests do not link to any shared libraries.
Run the tests for -musl targets
Find a solution for x86_64 targets. I'm leaning towards merging PR #1111 so that we wouldn't have a real need for poly1305_vec.c, and then removing poly1305_vec.c. Then we can remove the exception for x86_64 targets in build.rs and we should be good to go.

Answer 9 · 2020-12-03T19:05:19.000Z

PR #1133 shows how to use RUSTFLAGS="-Clink-self-contained=yes -Clinker=rust-lld" for linking al -musl targets. For x86_64 (only) we still need a sysroot with header files; for other architectures this should be working fine.

Answer 10 · 2020-12-10T14:35:26.000Z

I doubt using gcc-multilib with musl is the right thing to do but I could be convinced otherwise. I'll try experimenting with using --nostdlibinc and/or -nostdlib but it isn't clear that musl libc supports that configuration either.
Help understanding and/or resolving this is appreciated. Otherwise, maybe this is a good motivation to get rid of all the remaining C code in ring.

Hi,
I'm not sure if it will solve this problem, but in case it can help: I have no problem compiling dependencies relying on rustls (and thus ring) for the x86_64-unknown-linux-musl target from a rust:latest Docker image by running apt update && apt install -y musl-tools (https://github.com/skerkour/archive/blob/4a408405978ae714a7be9e92e2105324339f1f7b/simpleserver-rs/Dockerfile#L8)

Answer 11 · 2020-12-10T17:35:25.000Z

I'm not sure if it will solve this problem, but in case it can help: I have no problem compiling dependencies relying on rustls (and thus ring) for the x86_64-unknown-linux-musl target from a rust:latest Docker image by running apt update && apt install -y musl-tools (https://github.com/skerkour/archive/blob/4a408405978ae714a7be9e92e2105324339f1f7b/simpleserver-rs/Dockerfile#L8)

Are you then using the TARGET_CC=musl-gcc TARGET_AR=musl-ar or similar? Are you using RUSTFLAGS="-Clink-self-contained=yes -Clinker=rust-lld" for linking?

Answer 12 · 2020-12-10T17:47:17.000Z

No flags are used, the build command is as simple as cargo build --target x86_64-unknown-linux-musl --release

Are those flags special ring requirements, or are they workarounds to have MUSL compile?

Answer 13 · 2020-12-14T19:12:55.000Z

No flags are used, the build command is as simple as cargo build --target x86_64-unknown-linux-musl --release

Are those flags special ring requirements, or are they workarounds to have MUSL compile?

cc-rs will automatically use the musl tools (musl-cc) by default, so if you are happy with using musl-tools, then you will get the correct results.

If you want to use clang and rust-lld and "self contained" linking instead, then you do have to set up some things as described in ring's mk/cargo.sh.

Maybe I should change ring's CI/CD to test the "Just use the defaults" code path.

Answer 14 · 2021-11-19T13:44:11.000Z

Just want to add that if static linking is the only cencern, I'm able to produce a static linked binary using RUSTFLAGS="-C target-feature=+crt-static" cargo build --target x86_64-unknown-linux-gnu --release with glibc. I only use a subset of functions of ring though (namely hmac). Not sure would it affect linking.

Answer 15 · 2022-10-13T17:39:10.000Z

Hi @briansmith, any news on this issue?

Answer 16 · 2023-10-31T00:30:00.000Z

This has been working for a long time, AFAICT. We have aarch64-unknown-linux-musl, armv7-unknown-linux-musleabihf, i686-unknown-linux-musl, and x86_64-unknown-linux-musl tested in every CI run.