error: building the downstream libh3 C library using the Solana SDK via cmake
johhnsmmith198 opened this issue · 23 comments
Hi,
solana has this helloworld program that you can build (and deploy on a validator): https://github.com/solana-labs/example-helloworld
It works well before adding the h3ron library.
adding this dependency:
h3ron = "0.13.0"
and running:
npm run build:program-rust
would generate this error
Error: Function _ZN15crossbeam_epoch8internal6Global7collect17hdc7ca3f69bd0b12bE Stack offset of -4144 exceeded max offset of -4096 by 48 bytes, please minimize large stack variables
*** stack smashing detected ***: terminated
error: could not compile `crossbeam-epoch`
Caused by:
process didn't exit successfully: `rustc --crate-name crossbeam_epoch --edition=2018 /home/pc/.cargo/registry/src/github.com-1ecc6299db9ec823/crossbeam-epoch-0.9.5/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg 'feature="alloc"' --cfg 'feature="lazy_static"' --cfg 'feature="std"' -C metadata=1bb560ac1e04b0d7 -C extra-filename=-1bb560ac1e04b0d7 --out-dir /home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/deps --target bpfel-unknown-unknown -L dependency=/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/deps -L dependency=/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/release/deps --extern cfg_if=/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/deps/libcfg_if-1643e4a2c6efb510.rmeta --extern crossbeam_utils=/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/deps/libcrossbeam_utils-c2f18398aedce3c4.rmeta --extern lazy_static=/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/deps/liblazy_static-327a97971696967c.rmeta --extern memoffset=/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/deps/libmemoffset-1762d3769b1ca8cc.rmeta --extern scopeguard=/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/deps/libscopeguard-83203493741a9def.rmeta --cap-lints allow -C lto=no` (signal: 6, SIGABRT: process abort signal)
warning: build failed, waiting for other jobs to finish...
error: build failed
Could you please help?
Hi,
Seems this is caused by the crossbeam-epoch crate. h3ron does not use that library directly but only indirectly via rayon. There was a bug until recently (see #29) where h3ron pulled in rayon regardless if the use-rayon feature was activated. This should now be fixed in the master branch, but has not been released yet.
Try switching your dependency from the crates.io version "0.13" to latest master from git and see if the problem persists.
Thanks for your help
I gave it a try by changing this on the Cargo.toml
h3ron = { git = "https://github.com/nmandery/h3ron.git"}
I get this error now
Compiling h3ron-h3-sys v0.12.0 (https://github.com/nmandery/h3ron.git#ab3a27f3)
error: failed to run custom build command for `h3ron-h3-sys v0.12.0 (https://github.com/nmandery/h3ron.git#ab3a27f3)`
Caused by:
process didn't exit successfully: `/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/release/build/h3ron-h3-sys-1ab6cd26d7e5abe9/build-script-build` (exit status: 101)
--- stdout
cargo:rerun-if-changed=libh3
CMAKE_TOOLCHAIN_FILE_bpfel-unknown-unknown = None
CMAKE_TOOLCHAIN_FILE_bpfel_unknown_unknown = None
TARGET_CMAKE_TOOLCHAIN_FILE = None
CMAKE_TOOLCHAIN_FILE = None
CMAKE_GENERATOR_bpfel-unknown-unknown = None
CMAKE_GENERATOR_bpfel_unknown_unknown = None
TARGET_CMAKE_GENERATOR = None
CMAKE_GENERATOR = None
CMAKE_PREFIX_PATH_bpfel-unknown-unknown = None
CMAKE_PREFIX_PATH_bpfel_unknown_unknown = None
TARGET_CMAKE_PREFIX_PATH = None
CMAKE_PREFIX_PATH = None
CMAKE_bpfel-unknown-unknown = None
CMAKE_bpfel_unknown_unknown = None
TARGET_CMAKE = None
CMAKE = None
running: "cmake" "/home/pc/.cargo/git/checkouts/h3ron-31217b227e8cd3a6/ab3a27f/h3ron-h3-sys/libh3" "-DBUILD_BENCHMARKS=OFF" "-DBUILD_FILTERS=OFF" "-DBUILD_GENERATORS=OFF" "-DBUILD_TESTING=OFF" "-DENABLE_COVERAGE=OFF" "-DENABLE_DOCS=OFF" "-DENABLE_FORMAT=OFF" "-DENABLE_LINTING=OFF" "-DCMAKE_INSTALL_PREFIX=/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown" "-DCMAKE_C_COMPILER=/home/pc/.local/share/solana/install/releases/1.9.2/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC" "-DCMAKE_CXX_COMPILER=/usr/bin/c++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown" "-DCMAKE_ASM_COMPILER=/home/pc/.local/share/solana/install/releases/1.9.2/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang" "-DCMAKE_BUILD_TYPE=Release"
-- The C compiler identification is Clang 13.0.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - failed
-- Check for working C compiler: /home/pc/.local/share/solana/install/releases/1.9.2/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang
-- Check for working C compiler: /home/pc/.local/share/solana/install/releases/1.9.2/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang - broken
-- Configuring incomplete, errors occurred!
See also "/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeOutput.log".
See also "/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeError.log".
--- stderr
CMake Error at /usr/local/share/cmake-3.22/Modules/CMakeTestCCompiler.cmake:69 (message):
The C compiler
"/home/pc/.local/share/solana/install/releases/1.9.2/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang"
is not able to compile a simple test program.
It fails with the following output:
Change Dir: /home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeTmp
Run Build Command(s):/usr/bin/make -f Makefile cmTC_006a3/fast && /usr/bin/make -f CMakeFiles/cmTC_006a3.dir/build.make CMakeFiles/cmTC_006a3.dir/build
make[1]: Entering directory '/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_006a3.dir/testCCompiler.c.o
/home/pc/.local/share/solana/install/releases/1.9.2/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown -std=gnu99 -MD -MT CMakeFiles/cmTC_006a3.dir/testCCompiler.c.o -MF CMakeFiles/cmTC_006a3.dir/testCCompiler.c.o.d -o CMakeFiles/cmTC_006a3.dir/testCCompiler.c.o -c /home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeTmp/testCCompiler.c
Linking C executable cmTC_006a3
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/cmTC_006a3.dir/link.txt --verbose=1
/home/pc/.local/share/solana/install/releases/1.9.2/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown -rdynamic CMakeFiles/cmTC_006a3.dir/testCCompiler.c.o -o cmTC_006a3
/usr/bin/ld: unknown architecture of input file `CMakeFiles/cmTC_006a3.dir/testCCompiler.c.o' is incompatible with i386:x86-64 output
collect2: error: ld returned 1 exit status
clang-13: error: linker (via gcc) command failed with exit code 1 (use -v to see invocation)
make[1]: *** [CMakeFiles/cmTC_006a3.dir/build.make:100: cmTC_006a3] Error 1
make[1]: Leaving directory '/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeTmp'
make: *** [Makefile:127: cmTC_006a3/fast] Error 2
CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
CMakeLists.txt:59 (project)
thread 'main' panicked at '
command did not execute successfully, got: exit status: 1
build script failed, must exit now', /home/pc/.cargo/registry/src/github.com-1ecc6299db9ec823/cmake-0.1.46/src/lib.rs:974:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: build failed
Just so you know this is what the npm command that I am running is doing (cargo build-bpf):
npm run build:program-rust
> helloworld@0.0.1 build:program-rust /home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld
> cargo build-bpf --manifest-path=./src/program-rust/Cargo.toml --bpf-out-dir=dist/program
BPF SDK: /home/pc/.local/share/solana/install/releases/1.9.2/solana-release/bin/sdk/bpf
cargo-build-bpf child: rustup toolchain list -v
cargo-build-bpf child: cargo +bpf build --target bpfel-unknown-unknown --release
Compiling proc-macro2 v1.0.27
Compiling unicode-xid v0.2.2
Compiling syn v1.0.73
Compiling version_check v0.9.3
Compiling serde v1.0.126
...
Seems the crossbeam issue is solved - great.
Now the problem is building the downstream libh3 C library using the Solana SDK via cmake. It appears cmake correctly detects the C compiler of the SDK, but is not able to find a suitable linker. So it falls back to your systems default linker, but that one does not support the bpfel architecture. I suppose there should be a suitable linker with in the SDK in the same directory as the compiler found by cmake.
Messing with cmake can be tricky, but https://stackoverflow.com/questions/1867745/cmake-use-a-custom-linker gives some hints. The -D flags can also be set using environment variables like https://cmake.org/cmake/help/latest/envvar/CFLAGS.html
@nmandery I think I am getting onto something.
I was trying to step back and build and run the original h3 C library.
I was getting into some issues while running the example and created an issue for that: uber/h3#566
Someone was able to find that the linker needed -lh3 but also -lm for the math library.
Could this be related to my other linker issue above ?
Also I am seeing that I sudo make install the library called h3 but it seems h3ron is calling it libh3 (that I didn't install actually).
The h3ron-h3-sys crate compiles its own copy of the h3 library when the crate is build. This happens here https://github.com/nmandery/h3ron/blob/master/h3ron-h3-sys/build.rs#L15 and is done using cmake. The reason for this is to be able to do a static linking of the rust code to the C library and to be independent of h3 being installed with the system libraries.
That means all information about the environment we are compiling for must go there. That works great for the Solana-specfic compiler as this is set in the CC environment variable. So cmake finds this and uses this. But there is not such thing for the linker.
Please have a look in /home/pc/.local/share/solana/install/releases/1.9.2/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/ - there should be something called *ld*, thats the linker we need here. As this is LLVM, its certainly called lld.
cmake supports the CFLAGS environment variable, so we can use this to pass this information to cmake.
First run cargo clean in your project as cmake does lots of caching and will probably not use the new environment variable as long as the caches exist.
Try to build your project with something like
cargo clean
export CFLAGS="-fuse-ld=/home/pc/.local/share/solana/install/releases/1.9.2/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/lld"
npm run build:program-rustThats a quite long chain of build tools calling build tools - so hopefully this is of help to you.
libm should be contained in the LLVM of the SDK, I suppose.
Some related links:
I have switch to use 1.9.4 I hope its ok.
I double checked the paths
=> ls /home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin
clang clang++ clang-13 clang-cl clang-cpp ld64.lld ld.lld llc lld lld-link llvm-ar llvm-objcopy llvm-objdump llvm-readelf llvm-readobj
all good
npm run clean
export CFLAGS="-fuse-ld=/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/lld"
npm run build:program-rust
the output
Run Build Command(s):/usr/bin/make -f Makefile cmTC_8b0c0/fast && /usr/bin/make -f CMakeFiles/cmTC_8b0c0.dir/build.make CMakeFiles/cmTC_8b0c0.dir/build
make[1]: Entering directory '/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_8b0c0.dir/testCCompiler.c.o
/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown -fuse-ld=/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/lld -std=gnu99 -MD -MT CMakeFiles/cmTC_8b0c0.dir/testCCompiler.c.o -MF CMakeFiles/cmTC_8b0c0.dir/testCCompiler.c.o.d -o CMakeFiles/cmTC_8b0c0.dir/testCCompiler.c.o -c /home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeTmp/testCCompiler.c
clang-13: warning: argument unused during compilation: '-fuse-ld=/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/lld' [-Wunused-command-line-argument]
Linking C executable cmTC_8b0c0
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/cmTC_8b0c0.dir/link.txt --verbose=1
/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown -fuse-ld=/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/lld -rdynamic CMakeFiles/cmTC_8b0c0.dir/testCCompiler.c.o -o cmTC_8b0c0
gcc: error: unrecognized command line option ‘-fuse-ld=/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/lld’
clang-13: error: linker (via gcc) command failed with exit code 1 (use -v to see invocation)
make[1]: *** [CMakeFiles/cmTC_8b0c0.dir/build.make:100: cmTC_8b0c0] Error 1
make[1]: Leaving directory '/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeTmp'
make: *** [Makefile:127: cmTC_8b0c0/fast] Error 2
it says: argument unused during compilation
Nice that clang gives us this warning.
Try to use the LDFLAGS (https://cmake.org/cmake/help/latest/envvar/LDFLAGS.html) environment variable instead of CFLAGS. Dont forget the cargo clean
Thanks all good I am cleaning each time with npm run clean
I see it's clean because it restart downloading everything from scratch
> cargo build-bpf --manifest-path=./src/program-rust/Cargo.toml --bpf-out-dir=dist/program
BPF SDK: /home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf
cargo-build-bpf child: rustup toolchain list -v
cargo-build-bpf child: cargo +bpf build --target bpfel-unknown-unknown --release
Compiling proc-macro2 v1.0.27
Compiling unicode-xid v0.2.2
Compiling syn v1.0.73
...
I have open a new window to get rid of the CFLAGS and only have the LDFLAGS defined
same
gcc: error: unrecognized command line option ‘-fuse-ld=/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/lld’
clang-13: error: linker (via gcc) command failed with exit code 1 (use -v to see invocation)
also this is the beginning of the trace. I see clang - broken
Compiling h3ron-h3-sys v0.12.0 (https://github.com/nmandery/h3ron.git#ab3a27f3)
error: failed to run custom build command for h3ron-h3-sys v0.12.0 (https://github.com/nmandery/h3ron.git#ab3a27f3)
Caused by:
process didn't exit successfully: /home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/release/build/h3ron-h3-sys-1ab6cd26d7e5abe9/build-script-build (exit status: 101)
--- stdout
cargo:rerun-if-changed=libh3
CMAKE_TOOLCHAIN_FILE_bpfel-unknown-unknown = None
CMAKE_TOOLCHAIN_FILE_bpfel_unknown_unknown = None
TARGET_CMAKE_TOOLCHAIN_FILE = None
CMAKE_TOOLCHAIN_FILE = None
CMAKE_GENERATOR_bpfel-unknown-unknown = None
CMAKE_GENERATOR_bpfel_unknown_unknown = None
TARGET_CMAKE_GENERATOR = None
CMAKE_GENERATOR = None
CMAKE_PREFIX_PATH_bpfel-unknown-unknown = None
CMAKE_PREFIX_PATH_bpfel_unknown_unknown = None
TARGET_CMAKE_PREFIX_PATH = None
CMAKE_PREFIX_PATH = None
CMAKE_bpfel-unknown-unknown = None
CMAKE_bpfel_unknown_unknown = None
TARGET_CMAKE = None
CMAKE = None
running: "cmake" "/home/pc/.cargo/git/checkouts/h3ron-31217b227e8cd3a6/ab3a27f/h3ron-h3-sys/libh3" "-DBUILD_BENCHMARKS=OFF" "-DBUILD_FILTERS=OFF" "-DBUILD_GENERATORS=OFF" "-DBUILD_TESTING=OFF" "-DENABLE_COVERAGE=OFF" "-DENABLE_DOCS=OFF" "-DENABLE_FORMAT=OFF" "-DENABLE_LINTING=OFF" "-DCMAKE_INSTALL_PREFIX=/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown" "-DCMAKE_C_COMPILER=/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC" "-DCMAKE_CXX_COMPILER=/usr/bin/c++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown" "-DCMAKE_ASM_COMPILER=/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang" "-DCMAKE_BUILD_TYPE=Release"
-- The C compiler identification is Clang 13.0.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - failed
-- Check for working C compiler: /home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang
-- Check for working C compiler: /home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang - broken
-- Configuring incomplete, errors occurred!
gcc calling clang? Things are getting confusing ;) Also interesting that the flag is not understood.
One more attempt, if this does not work, I am slowly running out of ideas:
unset CFLAGS
unset LDFLAGS
# search in the SDK first
export PATH="/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/:$PATH"
cargo clean
# ... then buildThat bpf.mk file here could give more hints on expected environment settings for C/C++ builds to succeed: https://docs.solana.com/developing/on-chain-programs/developing-c#project-layout
well back to the beginning I think. Thanks for your help, really appreciate it
Run Build Command(s):/usr/bin/make -f Makefile cmTC_b4485/fast && /usr/bin/make -f CMakeFiles/cmTC_b4485.dir/build.make CMakeFiles/cmTC_b4485.dir/build
make[1]: Entering directory '/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_b4485.dir/testCCompiler.c.o
/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown -std=gnu99 -MD -MT CMakeFiles/cmTC_b4485.dir/testCCompiler.c.o -MF CMakeFiles/cmTC_b4485.dir/testCCompiler.c.o.d -o CMakeFiles/cmTC_b4485.dir/testCCompiler.c.o -c /home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeTmp/testCCompiler.c
Linking C executable cmTC_b4485
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/cmTC_b4485.dir/link.txt --verbose=1
/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown -rdynamic CMakeFiles/cmTC_b4485.dir/testCCompiler.c.o -o cmTC_b4485
/usr/bin/ld: unknown architecture of input file `CMakeFiles/cmTC_b4485.dir/testCCompiler.c.o' is incompatible with i386:x86-64 output
collect2: error: ld returned 1 exit status
clang-13: error: linker (via gcc) command failed with exit code 1 (use -v to see invocation)
make[1]: *** [CMakeFiles/cmTC_b4485.dir/build.make:100: cmTC_b4485] Error 1
make[1]: Leaving directory '/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeTmp'
make: *** [Makefile:127: cmTC_b4485/fast] Error 2
Solana allows you to build C program or Rust program. This would be for the C program.
https://github.com/solana-labs/example-helloworld/blob/master/src/program-c/makefile
When I say program I mean the solana smart contract
No worries.
I have to confess I do not understand much of the blockchain terminology ;)
Looking at https://github.com/solana-labs/solana/blob/master/sdk/bpf/c/bpf.mk it seems to contain the flags needed to compile C code for the pbf architecture. Maybe the compilation here works when you extract flags from the files BPF_C_FLAGS and BPF_LLD_FLAGS and put them in CFLAGS and LDFLAGS.
In case that gives conflicts with the rust compilation, maybe putting them directly into h3ron-h3-sys build.rs allows to circumvent that.
Thats the command to compile C for solana: https://github.com/solana-labs/solana/blob/master/sdk/bpf/c/bpf.mk#L159
and thats how it appears to be linked: https://github.com/solana-labs/solana/blob/master/sdk/bpf/c/bpf.mk#L187
In the example you linked that bpf.mk file gets included and applied to compile the example. So the solution is somewhere in there.
@nmandery
I decided to step back a little. I cloned h3ron and then ran those commands
export PATH="/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/:$PATH"
➜ funspace cd h3ron
➜ h3ron git:(master) ls
Cargo.toml CHANGES.md CONTRIBUTING.md data h3ron h3ron-graph h3ron-h3-sys h3ron-ndarray justfile LICENSE-MIT README.md
➜ h3ron git:(master) mkdir build
➜ h3ron git:(master) cd build
➜ build git:(master) "cmake" "/home/pc/.cargo/git/checkouts/h3ron-31217b227e8cd3a6/ab3a27f/h3ron-h3-sys/libh3" \
"-DBUILD_BENCHMARKS=OFF" \
"-DBUILD_FILTERS=OFF" \
"-DBUILD_GENERATORS=OFF" \
"-DBUILD_TESTING=OFF" \
"-DENABLE_COVERAGE=OFF" \
"-DENABLE_DOCS=OFF" \
"-DENABLE_FORMAT=OFF" \
"-DENABLE_LINTING=OFF" \
"-DCMAKE_INSTALL_PREFIX=/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out" \
"-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-none -v -fuse-ld=lld" \
"-DCMAKE_C_COMPILER=/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang" \
"-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC " \
"-DCMAKE_CXX_COMPILER=/usr/bin/c++" \
"-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown" \
"-DCMAKE_ASM_COMPILER=/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang" \
"-DCMAKE_BUILD_TYPE=Release"
-- The C compiler identification is Clang 13.0.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - failed
-- Check for working C compiler: /home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang
-- Check for working C compiler: /home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang - broken
CMake Error at /usr/local/share/cmake-3.22/Modules/CMakeTestCCompiler.cmake:69 (message):
The C compiler
"/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang"
is not able to compile a simple test program.
It fails with the following output:
Change Dir: /home/pc/Documents/funspace/h3ron/CMakeFiles/CMakeTmp
Run Build Command(s):/usr/bin/make -f Makefile cmTC_dab46/fast && /usr/bin/make -f CMakeFiles/cmTC_dab46.dir/build.make CMakeFiles/cmTC_dab46.dir/build
make[1]: Entering directory '/home/pc/Documents/funspace/h3ron/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_dab46.dir/testCCompiler.c.o
/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-none -v -fuse-ld=lld -std=gnu99 -MD -MT CMakeFiles/cmTC_dab46.dir/testCCompiler.c.o -MF CMakeFiles/cmTC_dab46.dir/testCCompiler.c.o.d -o CMakeFiles/cmTC_dab46.dir/testCCompiler.c.o -c /home/pc/Documents/funspace/h3ron/CMakeFiles/CMakeTmp/testCCompiler.c
clang version 13.0.0 (https://github.com/solana-labs/llvm-project.git 9a51f4037220994af74488ba064efc4e96ee08c0)
Target: bpfel-unknown-none
Thread model: posix
InstalledDir: /home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin
clang-13: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
(in-process)
"/home/pc/.cache/solana/v1.21/bpf-tools/llvm/bin/clang-13" -cc1 -triple bpfel-unknown-none -emit-obj -mrelax-all --mrelax-relocations -disable-free -disable-llvm-verifier -discard-value-names -main-file-name testCCompiler.c -mrelocation-model pic -pic-level 2 -fhalf-no-semantic-interposition -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -debugger-tuning=gdb -v -ffunction-sections -fdata-sections -fcoverage-compilation-dir=/home/pc/Documents/funspace/h3ron/CMakeFiles/CMakeTmp -resource-dir /home/pc/.cache/solana/v1.21/bpf-tools/llvm/lib/clang/13.0.0 -dependency-file CMakeFiles/cmTC_dab46.dir/testCCompiler.c.o.d -MT CMakeFiles/cmTC_dab46.dir/testCCompiler.c.o -sys-header-deps -std=gnu99 -fdebug-compilation-dir=/home/pc/Documents/funspace/h3ron/CMakeFiles/CMakeTmp -ferror-limit 19 -fgnuc-version=4.2.1 -faddrsig -o CMakeFiles/cmTC_dab46.dir/testCCompiler.c.o -x c /home/pc/Documents/funspace/h3ron/CMakeFiles/CMakeTmp/testCCompiler.c
clang -cc1 version 13.0.0 based upon LLVM 13.0.0-rust-dev default target x86_64-unknown-linux-gnu
#include "..." search starts here:
#include <...> search starts here:
/usr/local/include
/home/pc/.cache/solana/v1.21/bpf-tools/llvm/lib/clang/13.0.0/include
/usr/include
End of search list.
Linking C executable cmTC_dab46
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/cmTC_dab46.dir/link.txt --verbose=1
/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown -v -fuse-ld=lld -rdynamic CMakeFiles/cmTC_dab46.dir/testCCompiler.c.o -o cmTC_dab46
clang version 13.0.0 (https://github.com/solana-labs/llvm-project.git 9a51f4037220994af74488ba064efc4e96ee08c0)
Target: bpfel-unknown-none
Thread model: posix
InstalledDir: /home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin
"/usr/bin/gcc" -fuse-ld=lld -rdynamic -o cmTC_dab46 CMakeFiles/cmTC_dab46.dir/testCCompiler.c.o
ld.lld: error: CMakeFiles/cmTC_dab46.dir/testCCompiler.c.o is incompatible with elf64-x86-64
collect2: error: ld returned 1 exit status
clang-13: error: linker (via gcc) command failed with exit code 1 (use -v to see invocation)
make[1]: *** [CMakeFiles/cmTC_dab46.dir/build.make:100: cmTC_dab46] Error 1
make[1]: Leaving directory '/home/pc/Documents/funspace/h3ron/CMakeFiles/CMakeTmp'
make: *** [Makefile:127: cmTC_dab46/fast] Error 2
CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
CMakeLists.txt:59 (project)
-- Configuring incomplete, errors occurred!
See also "/home/pc/Documents/funspace/h3ron/build/CMakeFiles/CMakeOutput.log".
See also "/home/pc/Documents/funspace/h3ron/build/CMakeFiles/CMakeError.log".
running the same cmake command that solana is doing is triggering the error.
meaning its not really solana or rust the problem, it is really this cmake command that doesn't work with h3ron
Note I added -v -fuse-ld=lld to the DCMAKE_C_FLAGS
Would it help when we implement a feature-flag to compile libh3 without cmake using the cc crate?
@nmandery I would like to ask a few questions.
When a rust program add h3ron as a dependency. h3ron is being downloaded and a command is ran to build the underlying libh3 C library.
Where in your code is this building happening?
How can the parent program (in this case example-helloworld ) can pass their own variables like -DCMAKE_C_COMPILER -DCMAKE_ASM_FLAGS etc... to build the underlying c bindings library?
In this case, it seems that solana want the underlying c library to be built with BPF right ?
The libh3 C-sources are actually bundled in the h3ron-h3-sys crate when its downloaded from crates.io. The build of the C sources is done by the build.rs of the h3ron-h3-sys crate.
There is no possibility to pass custom flags to the cmake command - I am not aware of any way to do such things using cargo. The reason cmake is often used is that it figures out the correct settings for the host-system itself. Thats also the reason why cmake compiles the "simple test program" (see the cmake output) - it does so simply to validate its able to build. The basis for cmake being able to find the compiler etc are environment variables - these here are the supported ones. These can be used to pass information to cmake in a way which should also work with cargo being in between. On your system for example CC seems to be set so cmake finds the clang of the solana SDK. The environment-variable for the ASM_FLAGS seems to be this one: https://cmake.org/cmake/help/latest/envvar/ASM_DIALECTFLAGS.html
The tricky part here is that cmake is not supposed to build for the host system, but instead should be cross compile for solana. CMake has documentation how to use it for cross compilation: https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html#cross-compiling-toolchain - setting CMAKE_CROSSCOMPILING to disable the compiler checks could be a step in the right direction.
In case this can not be resolved using environment variables, perhaps forking the repo and modifying the build.rs file directly to make it work with solana would be an appropriate solution.
@nmandery thanks for your help.
What would the build.rs look like if I was using the system h3 that I built and installed previously, instead of rebuilding it?
You could leave out lines 12 and 15 - everything else you will still need though. You also got to change the dst_path to point to your manually build library and h3api.h header. The println!("cargo:rus.... statements give instructions to rustc on how to link the crate - see the documentation here.
You can choose between two kinds of linkage, static and dynamic. I am using static linking to produce a single binary with libh3 being contained within. I am also not sure if solana and bpf support dynamic linking.