rust-lang/wg-cargo-std-aware

Unable to build executable for musl target

HenryJk opened this issue · 10 comments

HenryJk commented

Project folder

fannkuchredux2.zip

Build Command

cargo +nightly build -Z build-std=std,panic_abort --target=x86_64-unknown-linux-musl --release

Error Message

error: linking with `cc` failed: exit status: 1
  |
  = note: "cc" "-m64" "/usr/lib/musl/lib/rcrt1.o" "/usr/lib/musl/lib/crti.o" "crtbeginS.o" "/home/henryj/Documents/Projects/salsa_rust/fannkuchredux/target/x86_64-unknown-linux-musl/release/deps/fannkuchredux-b4e9ec2a3b6be5b2.fannkuchredux.b2f0ac22-cgu.0.rcgu.o" "-Wl,--as-needed" "-L" "/home/henryj/Documents/Projects/salsa_rust/fannkuchredux/target/x86_64-unknown-linux-musl/release/deps" "-L" "/home/henryj/Documents/Projects/salsa_rust/fannkuchredux/target/release/deps" "-L" "/usr/lib/musl/lib" "-L" "/home/henryj/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib" "-Wl,--start-group" "-Wl,-Bstatic" "-lunwind" "/tmp/rustcp0cb3G/liblibc-3b1e88dbfd6f84e6.rlib" "-Wl,--end-group" "/home/henryj/Documents/Projects/salsa_rust/fannkuchredux/target/x86_64-unknown-linux-musl/release/deps/libcompiler_builtins-49455d93a82e9390.rlib" "-Wl,-Bdynamic" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-nostartfiles" "-L" "/home/henryj/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib" "-L" "/home/henryj/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained" "-o" "/home/henryj/Documents/Projects/salsa_rust/fannkuchredux/target/x86_64-unknown-linux-musl/release/deps/fannkuchredux-b4e9ec2a3b6be5b2" "-Wl,--gc-sections" "-static-pie" "-Wl,-zrelro,-znow" "-Wl,-O1" "-nodefaultlibs" "crtendS.o" "/usr/lib/musl/lib/crtn.o"
  = note: /usr/bin/ld: cannot find crtbeginS.o: No such file or directory
          /usr/bin/ld: cannot find -lunwind
          collect2: error: ld returned 1 exit status

Operating system

Arch Linux

Cargo version

cargo 1.58.0-nightly (ad50d0d26 2021-11-17)

12101111 commented

Those file should locate at /home/henryj/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained

ehuss commented

@12101111 Will this be fixed by rust-lang/rust#90681?

HenryJk commented

It can compile after I run rustup +nightly target add x86_64-unknown-linux-musl. However, I still need to put -L/home/henryj/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained on rustflags. I thought cargo link to system library on compilation, apparently it isn't and the library for stable and nightly has to be downloaded separately.

  • 👍1
12101111 commented

However, I still need to put -L/home/henryj/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained on rustflags.

It can be enable by rustc codegen option -Clink-self-contained=on, but it should enable by default for musl target. No idea why it don't work for you.

You can set it in .cargo/config.toml if you want a dynamic linked musl binary:

[build]
target = "x86_64-unknown-linux-musl"

[target.x86_64-unknown-linux-musl]
linker = "ld"
rustflags = ["-Ctarget-feature=-crt-static", "-Clink-self-contained=on", "-L/usr/lib/x86_64-linux-musl", "-Clink-args=--dynamic-linker /lib/ld-musl-x86_64.so.1"]

[unstable]
build-std = ["core", "compiler_builtins", "alloc", "std", "panic_abort"]
build-std-features = ["llvm-libunwind"]

Tested on Ubuntu 20.04:

> ls -la /lib/ld-musl-x86_64.so.1
lrwxrwxrwx 1 root root 25 Oct 14  2019 /lib/ld-musl-x86_64.so.1 -> x86_64-linux-musl/libc.so
> ls /usr/lib/x86_64-linux-musl/
Scrt1.o  crti.o  libc.a   libcrypt.a  libm.a        libresolv.a  libutil.a  rcrt1.o
crt1.o   crtn.o  libc.so  libdl.a     libpthread.a  librt.a      libxnet.a
> cargo +nightly build
...
> file target/x86_64-unknown-linux-musl/debug/hello
target/x86_64-unknown-linux-musl/debug/hello: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-x86_64.so.1, with debug_info, not stripped
> readelf -d target/x86_64-unknown-linux-musl/debug/hello

Dynamic section at offset 0xa2a88 contains 25 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libc.so]
 0x000000000000000c (INIT)               0xd000
 0x000000000000000d (FINI)               0x7c77f
 0x0000000000000019 (INIT_ARRAY)         0x9f988
 0x000000000000001b (INIT_ARRAYSZ)       8 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x9f990
 0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
 0x0000000000000004 (HASH)               0x2c8
 0x000000006ffffef5 (GNU_HASH)           0x450
 0x0000000000000005 (STRTAB)             0xa08
 0x0000000000000006 (SYMTAB)             0x498
 0x000000000000000a (STRSZ)              690 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000015 (DEBUG)              0x0
 0x0000000000000003 (PLTGOT)             0xa3c58
 0x0000000000000002 (PLTRELSZ)           288 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0xc378
 0x0000000000000007 (RELA)               0xcc0
 0x0000000000000008 (RELASZ)             46776 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x000000000000001e (FLAGS)              BIND_NOW
 0x000000006ffffffb (FLAGS_1)            Flags: NOW PIE
 0x000000006ffffff9 (RELACOUNT)          1906
 0x0000000000000000 (NULL)               0x0
> target/x86_64-unknown-linux-musl/debug/hello
Hello, world!

If you are try to build a static linked binary (which is default for musl target), you don't have to add "-L/usr/lib/x86_64-linux-musl" option or any other flags after rust-lang/rust#90681 is merged.

[build]
target = "x86_64-unknown-linux-musl"

[unstable]
build-std = ["core", "compiler_builtins", "alloc", "std", "panic_abort"]
HenryJk commented

Cargo can't find native libc.a (musl). However, the location for libc is different on Arch compared to Ubuntu. On Arch, it is installed at /usr/lib/musl/lib/. The location is also not normally linked by LD_LIBRARY_PATH. "-Clink-self-contained=on" option doesn't work and in the end, I still need to link to my toolchain folder like before.

My .cargo/config.toml:

[build]
jobs = 1
rustflags = ["-Ctarget-cpu=ivybridge", "-Clink-self-contained=on"]

My build command:

cargo +nightly build -Z build-std=std,panic_abort -Z build-std-features=panic_immediate_abort --target=x86_64-unknown-linux-musl --release

Error message:

    Compiling core v0.0.0 (/home/henryj/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core)
    Compiling rustc-std-workspace-core v1.99.0 (/home/henryj/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/rustc-std-workspace-core)
    Compiling compiler_builtins v0.1.52
    Compiling libc v0.2.106
error: could not find native static library `c`, perhaps an -L flag is missing?

error: could not compile `libc` due to previous error
kornelski commented

edit: it's fixed as of 1.77.0-nightly

I have the same problem on macOS. Adding "-L/Users/…/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/" to rustflags has solved it for me.

I'm using filosottile/musl-cross/musl-cross

 export CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_LINKER=x86_64-linux-gnu-gcc
 export TARGET_CC=x86_64-linux-musl-gcc
 cargo +nightly build -Z build-std=std,panic_abort -Z build-std-features=panic_immediate_abort --target x86_64-unknown-linux-musl --release
   = note: x86_64-linux-musl-gcc: error: rcrt1.o: No such file or directory
           x86_64-linux-musl-gcc: error: crti.o: No such file or directory
           x86_64-linux-musl-gcc: error: crtbeginS.o: No such file or directory
           x86_64-linux-musl-gcc: error: crtendS.o: No such file or directory
           x86_64-linux-musl-gcc: error: crtn.o: No such file or directory
c272 commented

Just had a go at reproducing this on my x86_64-unknown-linux-gnu host machine targeting both x86_64-unknown-linux-musl and aarch64-unknown-linux-musl on a recent toolchain, and couldn't seem to recreate the issue. Would you mind sharing any extra information about your environment/your .cargo/config.toml if possible, @kornelski?

The following options seemed to work for me (compiler-builtins-c required here for softfp symbols):
.cargo/config.toml

[target.aarch64-unknown-linux-musl]
linker = "aarch64-linux-musl-ld"

RUST_COMPILER_RT_ROOT=... cargo +tc build -Z build-std=std,panic_abort -Z build-std-features=compiler-builtins-c --target=aarch64-unknown-linux-musl

kornelski commented

@c272 I've ran the build again, and it's working now.

  • 🎉1
Techno-Fox commented

What's the status on this issue? I just had the same problem and it seemed that adding the -L/path/to/self-contained worked. But why do we have to add that manually?

Techno-Fox commented

Even weirder, just tried cargo cross (https://github.com/cross-rs/cross) with it, and it worked just fine without specified -L/path/to/self-contained