Rust toolchain broken with recent RootFS

Question

Rust toolchain broken with recent RootFS

Opened this issue 3 years ago · 2 comments

On master (tests in #138 reproduce the failure):

julia> BinaryBuilderBase.runshell(; compilers=[:c, :rust, :go]);
sandbox:${WORKSPACE} # export SUPER_VERBOSE=1
sandbox:${WORKSPACE} # make -j${nproc} -sC /usr/share/testsuite install
x86_64-linux-gnu-cc -o /tmp/testsuite/x86_64-linux-gnu/c/hello_world/hello_world -g -O2 hello_world.c
gcc -o /tmp/testsuite/x86_64-linux-gnu/c/openmp/openmp -g -O2 -fopenmp openmp.c
x86_64-linux-gnu-cc -shared -o /tmp/testsuite/x86_64-linux-gnu/c/dyn_link/libfoo/libfoo.so -fPIC -g -O2 -lm libfoo/libfoo.c
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc -D_GLIBCXX_USE_CXX11_ABI=1 -D_DARWIN_FEATURE_CLOCK_GETTIME=0 -frandom-seed=0x4df3e497 -march=x86-64 -mtune=generic -o /tmp/testsuite/x86_64-linux-gnu/c/openmp/openmp -g -O2 -fopenmp openmp.c
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc -D_GLIBCXX_USE_CXX11_ABI=1 -D_DARWIN_FEATURE_CLOCK_GETTIME=0 -frandom-seed=0x91d8f682 -march=x86-64 -mtune=generic -o /tmp/testsuite/x86_64-linux-gnu/c/hello_world/hello_world -g -O2 hello_world.c
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc -D_GLIBCXX_USE_CXX11_ABI=1 -D_DARWIN_FEATURE_CLOCK_GETTIME=0 -frandom-seed=0xefd9e6a4 -march=x86-64 -mtune=generic -shared -o /tmp/testsuite/x86_64-linux-gnu/c/dyn_link/libfoo/libfoo.so -fPIC -g -O2 -lm libfoo/libfoo.c
x86_64-linux-gnu-cc -o /tmp/testsuite/x86_64-linux-gnu/c/dyn_link/dyn_link -I/usr/share/testsuite/c/dyn_link/libfoo -fPIC -g -O2 -Wl,-z,origin -Wl,-rpath,$ORIGIN/libfoo -L/tmp/testsuite/x86_64-linux-gnu/c/dyn_link/libfoo -lfoo dyn_link.c
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc -D_GLIBCXX_USE_CXX11_ABI=1 -D_DARWIN_FEATURE_CLOCK_GETTIME=0 -frandom-seed=0xbd703e78 -march=x86-64 -mtune=generic -o /tmp/testsuite/x86_64-linux-gnu/c/dyn_link/dyn_link -I/usr/share/testsuite/c/dyn_link/libfoo -fPIC -g -O2 -Wl,-z,origin -Wl,-rpath,$ORIGIN/libfoo -L/tmp/testsuite/x86_64-linux-gnu/c/dyn_link/libfoo -lfoo dyn_link.c
x86_64-linux-gnu-c++ -shared -o /tmp/testsuite/x86_64-linux-gnu/cxx/dyn_link/librepeater/librepeater.so -fPIC -g -O2 -lm librepeater/librepeater.cc
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-g++ -D_GLIBCXX_USE_CXX11_ABI=1 -D_DARWIN_FEATURE_CLOCK_GETTIME=0 -frandom-seed=0x03419044 -march=x86-64 -mtune=generic -shared -o /tmp/testsuite/x86_64-linux-gnu/cxx/dyn_link/librepeater/librepeater.so -fPIC -g -O2 -lm librepeater/librepeater.cc
x86_64-linux-gnu-c++ -o /tmp/testsuite/x86_64-linux-gnu/cxx/hello_world/hello_world -g -O2 hello_world.cc
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-g++ -D_GLIBCXX_USE_CXX11_ABI=1 -D_DARWIN_FEATURE_CLOCK_GETTIME=0 -frandom-seed=0x405d8293 -march=x86-64 -mtune=generic -o /tmp/testsuite/x86_64-linux-gnu/cxx/hello_world/hello_world -g -O2 hello_world.cc
x86_64-linux-gnu-f77 -o /tmp/testsuite/x86_64-linux-gnu/fortran/hello_world/hello_world.o -c hello_world.f
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-gfortran -D_GLIBCXX_USE_CXX11_ABI=1 -D_DARWIN_FEATURE_CLOCK_GETTIME=0 -frandom-seed=0x1f7de1e0 -march=x86-64 -mtune=generic -o /tmp/testsuite/x86_64-linux-gnu/fortran/hello_world/hello_world.o -c hello_world.f
x86_64-linux-gnu-cc -o /tmp/testsuite/x86_64-linux-gnu/fortran/hello_world/hello_world -g -O2 /tmp/testsuite/x86_64-linux-gnu/fortran/hello_world/hello_world.o -lgfortran -lm -lquadmath
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc -D_GLIBCXX_USE_CXX11_ABI=1 -D_DARWIN_FEATURE_CLOCK_GETTIME=0 -frandom-seed=0x27fdff42 -march=x86-64 -mtune=generic -o /tmp/testsuite/x86_64-linux-gnu/fortran/hello_world/hello_world -g -O2 /tmp/testsuite/x86_64-linux-gnu/fortran/hello_world/hello_world.o -lgfortran -lm -lquadmath
x86_64-linux-gnu-f77 -o /tmp/testsuite/x86_64-linux-gnu/fortran/hello_world/hello_world hello_world.f
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-gfortran -D_GLIBCXX_USE_CXX11_ABI=1 -D_DARWIN_FEATURE_CLOCK_GETTIME=0 -frandom-seed=0x79b8f319 -march=x86-64 -mtune=generic -o /tmp/testsuite/x86_64-linux-gnu/fortran/hello_world/hello_world hello_world.f
x86_64-linux-gnu-go build -o /tmp/testsuite/x86_64-linux-gnu/go/float_abi/float_abi float_abi.go
/opt/x86_64-linux-musl/go/bin/go build -o /tmp/testsuite/x86_64-linux-gnu/go/float_abi/float_abi float_abi.go
x86_64-linux-gnu-c++ -o /tmp/testsuite/x86_64-linux-gnu/cxx/dyn_link/dyn_link -I/usr/share/testsuite/cxx/dyn_link/librepeater -fPIC -g -O2 -Wl,-z,origin -Wl,-rpath,$ORIGIN/librepeater -L/tmp/testsuite/x86_64-linux-gnu/cxx/dyn_link/librepeater -lrepeater dyn_link.cc
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-g++ -D_GLIBCXX_USE_CXX11_ABI=1 -D_DARWIN_FEATURE_CLOCK_GETTIME=0 -frandom-seed=0x6f690163 -march=x86-64 -mtune=generic -o /tmp/testsuite/x86_64-linux-gnu/cxx/dyn_link/dyn_link -I/usr/share/testsuite/cxx/dyn_link/librepeater -fPIC -g -O2 -Wl,-z,origin -Wl,-rpath,$ORIGIN/librepeater -L/tmp/testsuite/x86_64-linux-gnu/cxx/dyn_link/librepeater -lrepeater dyn_link.cc
x86_64-linux-gnu-go build -o /tmp/testsuite/x86_64-linux-gnu/go/hello_world/hello_world hello_world.go
/opt/x86_64-linux-musl/go/bin/go build -o /tmp/testsuite/x86_64-linux-gnu/go/hello_world/hello_world hello_world.go
cargo install --verbose --path . --root /tmp/testsuite/x86_64-linux-gnu/rust/cargo_build
/opt/x86_64-linux-musl/bin/cargo install --verbose --path . --root /tmp/testsuite/x86_64-linux-gnu/rust/cargo_build
  Installing cargo_build v0.1.0 (/usr/share/testsuite/rust/cargo_build)
x86_64-linux-gnu-rustc -o /tmp/testsuite/x86_64-linux-gnu/rust/hello_world/hello_world hello_world.rs
/opt/x86_64-linux-musl/bin/rustc --target=x86_64-unknown-linux-gnu -Clinker=x86_64-linux-gnu-gcc -o /tmp/testsuite/x86_64-linux-gnu/rust/hello_world/hello_world hello_world.rs
error: failed to compile `cargo_build v0.1.0 (/usr/share/testsuite/rust/cargo_build)`, intermediate artifacts can be found at `/usr/share/testsuite/rust/cargo_build/target`

Caused by:
  failed to run `rustc` to learn about target-specific information

Caused by:
  process didn't exit successfully: `x86_64-linux-gnu-rustc - --crate-name ___ --print=file-names --target x86_64-unknown-linux-gnu --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=cfg` (exit code: 1)
--- stderr
/opt/x86_64-linux-musl/bin/rustc --target=x86_64-unknown-linux-gnu -Clinker=x86_64-linux-gnu-gcc - --crate-name ___ --print=file-names --target x86_64-unknown-linux-gnu --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=cfg
error: Option 'target' given more than once


make[1]: *** [Makefile:7: /tmp/testsuite/x86_64-linux-gnu/rust/cargo_build/cargo_build] Error 101
make: *** [Makefile:48: install-project-rust-cargo_build] Error 2
make: *** Waiting for unfinished jobs....
sandbox:${WORKSPACE} #

With #137:

julia> BinaryBuilderBase.runshell(; compilers=[:c, :rust, :go]);
[ Info: Running privileged container via `sudo`, may ask for your password:
sandbox:${WORKSPACE} # export SUPER_VERBOSE=1
sandbox:${WORKSPACE} # make -j${nproc} -sC /usr/share/testsuite install
x86_64-linux-gnu-ccx86_64-linux-gnu-c++  x86_64-linux-gnu-f77-o /tmp/testsuite/x86_64-linux-gnu/c/hello_world/hello_world -g -O2 hello_world.c-o /tmp/testsuite/x86_64-linux-gnu/cxx/hello_world/hello_world -g -O2 hello_world.cc
 
-o /tmp/testsuite/x86_64-linux-gnu/fortran/hello_world/hello_world.o -c hello_world.f
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc -D_GLIBCXX_USE_CXX11_ABI=1 -D_DARWIN_FEATURE_CLOCK_GETTIME=0 -frandom-seed=0x91d8f682 -march=x86-64 -mtune=generic -o /tmp/testsuite/x86_64-linux-gnu/c/hello_world/hello_world -g -O2 hello_world.c
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-g++ -D_GLIBCXX_USE_CXX11_ABI=1 -D_DARWIN_FEATURE_CLOCK_GETTIME=0 -frandom-seed=0x405d8293 -march=x86-64 -mtune=generic -o /tmp/testsuite/x86_64-linux-gnu/cxx/hello_world/hello_world -g -O2 hello_world.cc
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-gfortran -D_GLIBCXX_USE_CXX11_ABI=1 -D_DARWIN_FEATURE_CLOCK_GETTIME=0 -frandom-seed=0x1f7de1e0 -march=x86-64 -mtune=generic -o /tmp/testsuite/x86_64-linux-gnu/fortran/hello_world/hello_world.o -c hello_world.f
x86_64-linux-gnu-cc -o /tmp/testsuite/x86_64-linux-gnu/fortran/hello_world/hello_world -g -O2 /tmp/testsuite/x86_64-linux-gnu/fortran/hello_world/hello_world.o -lgfortran -lm -lquadmath
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc -D_GLIBCXX_USE_CXX11_ABI=1 -D_DARWIN_FEATURE_CLOCK_GETTIME=0 -frandom-seed=0x27fdff42 -march=x86-64 -mtune=generic -o /tmp/testsuite/x86_64-linux-gnu/fortran/hello_world/hello_world -g -O2 /tmp/testsuite/x86_64-linux-gnu/fortran/hello_world/hello_world.o -lgfortran -lm -lquadmath
x86_64-linux-gnu-f77 -o /tmp/testsuite/x86_64-linux-gnu/fortran/hello_world/hello_world hello_world.f
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-gfortran -D_GLIBCXX_USE_CXX11_ABI=1 -D_DARWIN_FEATURE_CLOCK_GETTIME=0 -frandom-seed=0x79b8f319 -march=x86-64 -mtune=generic -o /tmp/testsuite/x86_64-linux-gnu/fortran/hello_world/hello_world hello_world.f
x86_64-linux-gnu-go build -o /tmp/testsuite/x86_64-linux-gnu/go/hello_world/hello_world hello_world.go
/opt/x86_64-linux-musl/go/bin/go build -o /tmp/testsuite/x86_64-linux-gnu/go/hello_world/hello_world hello_world.go
x86_64-linux-gnu-rustc -o /tmp/testsuite/x86_64-linux-gnu/rust/hello_world/hello_world hello_world.rs
/opt/x86_64-linux-musl/bin/rustc --target=x86_64-unknown-linux-gnu -Clinker=x86_64-linux-gnu-gcc -o /tmp/testsuite/x86_64-linux-gnu/rust/hello_world/hello_world hello_world.rs
x86_64-linux-gnu-cc -shared -o /tmp/testsuite/x86_64-linux-gnu/c/dyn_link/libfoo/libfoo.so -fPIC -g -O2 -lm libfoo/libfoo.c
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc -D_GLIBCXX_USE_CXX11_ABI=1 -D_DARWIN_FEATURE_CLOCK_GETTIME=0 -frandom-seed=0xefd9e6a4 -march=x86-64 -mtune=generic -shared -o /tmp/testsuite/x86_64-linux-gnu/c/dyn_link/libfoo/libfoo.so -fPIC -g -O2 -lm libfoo/libfoo.c
x86_64-linux-gnu-cc -o /tmp/testsuite/x86_64-linux-gnu/c/dyn_link/dyn_link -I/usr/share/testsuite/c/dyn_link/libfoo -fPIC -g -O2 -Wl,-z,origin -Wl,-rpath,$ORIGIN/libfoo -L/tmp/testsuite/x86_64-linux-gnu/c/dyn_link/libfoo -lfoo dyn_link.c
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc -D_GLIBCXX_USE_CXX11_ABI=1 -D_DARWIN_FEATURE_CLOCK_GETTIME=0 -frandom-seed=0xbd703e78 -march=x86-64 -mtune=generic -o /tmp/testsuite/x86_64-linux-gnu/c/dyn_link/dyn_link -I/usr/share/testsuite/c/dyn_link/libfoo -fPIC -g -O2 -Wl,-z,origin -Wl,-rpath,$ORIGIN/libfoo -L/tmp/testsuite/x86_64-linux-gnu/c/dyn_link/libfoo -lfoo dyn_link.c
x86_64-linux-gnu-c++ -shared -o /tmp/testsuite/x86_64-linux-gnu/cxx/dyn_link/librepeater/librepeater.so -fPIC -g -O2 -lm librepeater/librepeater.cc
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-g++ -D_GLIBCXX_USE_CXX11_ABI=1 -D_DARWIN_FEATURE_CLOCK_GETTIME=0 -frandom-seed=0x03419044 -march=x86-64 -mtune=generic -shared -o /tmp/testsuite/x86_64-linux-gnu/cxx/dyn_link/librepeater/librepeater.so -fPIC -g -O2 -lm librepeater/librepeater.cc
x86_64-linux-gnu-c++ -o /tmp/testsuite/x86_64-linux-gnu/cxx/dyn_link/dyn_link -I/usr/share/testsuite/cxx/dyn_link/librepeater -fPIC -g -O2 -Wl,-z,origin -Wl,-rpath,$ORIGIN/librepeater -L/tmp/testsuite/x86_64-linux-gnu/cxx/dyn_link/librepeater -lrepeater dyn_link.cc
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-g++ -D_GLIBCXX_USE_CXX11_ABI=1 -D_DARWIN_FEATURE_CLOCK_GETTIME=0 -frandom-seed=0x6f690163 -march=x86-64 -mtune=generic -o /tmp/testsuite/x86_64-linux-gnu/cxx/dyn_link/dyn_link -I/usr/share/testsuite/cxx/dyn_link/librepeater -fPIC -g -O2 -Wl,-z,origin -Wl,-rpath,$ORIGIN/librepeater -L/tmp/testsuite/x86_64-linux-gnu/cxx/dyn_link/librepeater -lrepeater dyn_link.cc
gcc -o /tmp/testsuite/x86_64-linux-gnu/c/openmp/openmp -g -O2 -fopenmp openmp.c
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc -D_GLIBCXX_USE_CXX11_ABI=1 -D_DARWIN_FEATURE_CLOCK_GETTIME=0 -frandom-seed=0x4df3e497 -march=x86-64 -mtune=generic -o /tmp/testsuite/x86_64-linux-gnu/c/openmp/openmp -g -O2 -fopenmp openmp.c
sandbox:${WORKSPACE} #

Answer 1 · 2021-04-21T23:48:51.000Z

Looks like when we invoke cargo, if RUSTC=rustc, it looks up the rustc binary next to it directly, but if RUSTC=x86_64-linux-musl-rustc (as it is set here) it ends up looping back through the wrapper and accidentally adding --target twice.

We should:

Not use := in that common.mk, but instead use ?= to properly inherit whatever is being set by the BB environment
Fix our rust wrapper to have a check to ensure that if --target is set, it is the correct one, and if so don't add --target. If it's set and is not the correct one, it should error. And of course if it is not set, it should continue to set --target.

Answer 2 · 2021-04-24T03:22:30.000Z

Not use := in that common.mk, but instead use ?= to properly inherit whatever is being set by the BB environment

JuliaPackaging/Yggdrasil#2891