Unable to build 0.11.1: warning _FORTIFY_SOURCE
kvark opened this issue · 19 comments
Full log:
[nix-shell:/x/code/naga/fuzz]$ cargo install afl
Updating crates.io index
Downloaded afl v0.11.1
Downloaded 1 crate (3.6 MB) in 0.76s
Installing afl v0.11.1
Compiling libc v0.2.112
Compiling semver v1.0.4
Compiling unicode-width v0.1.9
Compiling strsim v0.8.0
Compiling ansi_term v0.12.1
Compiling vec_map v0.8.2
Compiling bitflags v1.3.2
Compiling cc v1.0.72
Compiling textwrap v0.11.0
Compiling rustc_version v0.4.0
Compiling dirs-sys v0.3.6
Compiling atty v0.2.14
Compiling clap v2.34.0
Compiling dirs v3.0.2
Compiling xdg v2.4.0
Compiling afl v0.11.1
error: failed to run custom build command for `afl v0.11.1`
Caused by:
process didn't exit successfully: `/run/user/1000/cargo-install0HIprX/release/build/afl-e7a898d3022feeeb/build-script-build` (exit status: 101)
--- stdout
[*] Compiling afl++ for OS Linux on ARCH x86_64
Compiling DEBUG version of binaries
rm -f afl-fuzz afl-showmap afl-tmin afl-gotcpu afl-analyze libradamsa.so afl-fuzz-document afl-as as afl-g++ afl-clang afl-clang++ *.o src/*.o *~ a.out core core.[1-9][0-9]* *.stackdump .test .test1 .test2 test-instr .test-instr0 .test-instr1 afl-qemu-trace afl-gcc-fast afl-gcc-pass.so afl-g++-fast ld *.so *.8 test/unittests/*.o test/unittests/unit_maybe_alloc test/unittests/preallocable .afl-* afl-gcc afl-g++ afl-clang afl-clang++ test/unittests/unit_hash test/unittests/unit_rand
make -f GNUmakefile.llvm clean
make[1]: Entering directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus'
[+] llvm_mode detected llvm 10+, enabling neverZero implementation and c++14
[+] llvm_mode detected llvm 11+, enabling afl-lto LTO implementation
rm -f *.o *.so *~ a.out core core.[1-9][0-9]* .test2 test-instr .test-instr0 .test-instr1 *.dwo
rm -f ./afl-cc ./afl-compiler-rt.o ./afl-compiler-rt-32.o ./afl-compiler-rt-64.o ./afl-llvm-pass.so ./SanitizerCoveragePCGUARD.so ./split-compares-pass.so ./split-switches-pass.so ./cmplog-routines-pass.so ./cmplog-instructions-pass.so ./cmplog-switches-pass.so ./afl-llvm-dict2file.so ./compare-transform-pass.so ./afl-ld-lto ./afl-llvm-lto-instrumentlist.so ./afl-llvm-lto-instrumentation.so ./SanitizerCoverageLTO.so afl-common.o ./afl-c++ ./afl-lto ./afl-lto++ ./afl-clang-lto* ./afl-clang-fast* ./afl-clang*.8 ./ld ./afl-ld ./afl-llvm-rt*.o instrumentation/*.o
make[1]: Leaving directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus'
make -f GNUmakefile.gcc_plugin clean
make[1]: Entering directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus'
rm -f *.o *.so *~ a.out core core.[1-9][0-9]* test-instr .test-instr0 .test-instr1 .test2
rm -f ./afl-gcc-pass.so ./afl-compiler-rt.o ./afl-compiler-rt-32.o ./afl-compiler-rt-64.o afl-common.o ./afl-g++-fast ./afl-g*-fast.8 instrumentation/*.o
make[1]: Leaving directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus'
make -C utils/libdislocator clean
make[1]: Entering directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus/utils/libdislocator'
rm -f *.o *.so *~ a.out core core.[1-9][0-9]*
rm -f ../../libdislocator.so
make[1]: Leaving directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus/utils/libdislocator'
make -C utils/libtokencap clean
make[1]: Entering directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus/utils/libtokencap'
rm -f *.o *.so *~ a.out core core.[1-9][0-9]*
rm -fv ../../libtokencap.so
make[1]: Leaving directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus/utils/libtokencap'
make -C utils/aflpp_driver clean
make[1]: Entering directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus/utils/aflpp_driver'
rm -f *.o libAFLDriver*.a libAFLQemuDriver.a aflpp_qemu_driver_hook.so *~ core aflpp_driver_test
make[1]: Leaving directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus/utils/aflpp_driver'
make -C utils/afl_network_proxy clean
make[1]: Entering directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus/utils/afl_network_proxy'
rm -f afl-network-client afl-network-server *~ core
make[1]: Leaving directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus/utils/afl_network_proxy'
make -C utils/socket_fuzzing clean
make[1]: Entering directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus/utils/socket_fuzzing'
rm -f socketfuzz32.so socketfuzz64.so
make[1]: Leaving directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus/utils/socket_fuzzing'
make -C utils/argv_fuzzing clean
make[1]: Entering directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus/utils/argv_fuzzing'
rm -f argvfuzz32.so argvfuzz64.so
make[1]: Leaving directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus/utils/argv_fuzzing'
make -C qemu_mode/unsigaction clean
make[1]: Entering directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus/qemu_mode/unsigaction'
rm -f unsigaction.so
make[1]: Leaving directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus/qemu_mode/unsigaction'
make -C qemu_mode/libcompcov clean
make[1]: Entering directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus/qemu_mode/libcompcov'
rm -f *.o *.so *~ a.out core core.[1-9][0-9]*
rm -f ../../libcompcov.so compcovtest
make[1]: Leaving directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus/qemu_mode/libcompcov'
make -C qemu_mode/libqasan clean
make[1]: Entering directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus/qemu_mode/libqasan'
rm -f *.o *.so *~ a.out core core.[1-9][0-9]*
rm -f ../../libqasan.so
make[1]: Leaving directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus/qemu_mode/libqasan'
make -C frida_mode clean
make[1]: Entering directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus/frida_mode'
rm -rf /home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus/frida_mode/build/
make[1]: Leaving directory '/home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/AFLplusplus/frida_mode'
rm -rf qemu_mode/qemuafl
rm -rf unicorn_mode/unicornafl
[!] Note: skipping x86 compilation checks (AFL_NO_X86 set).
[-] shmat seems not to be working, switching to mmap implementation
[-] You seem to need to install the package python3-dev, python2-dev or python-dev (and perhaps python[23]-apt), but it is optional so we continue
[+] Everything seems to be working, ready to compile.
gcc -ggdb3 -O0 -Wall -Wextra -Werror -g -Wno-pointer-sign -Wno-variadic-macros -Wall -Wextra -Wpointer-arith -I include/ -DAFL_PATH=\"/lib/afl\" -DBIN_PATH=\"/bin\" -DDOC_PATH=\"/share/doc/afl\" -DUSEMMAP=1 -flto -c src/afl-common.c -o src/afl-common.o
--- stderr
GNUmakefile.llvm:68: you are using an in-development llvm version - this might break llvm_mode!
GNUmakefile.llvm:120: we have trouble finding clang - llvm-config is not helping us
GNUmakefile.llvm:135: we have trouble finding clang++ - llvm-config is not helping us
GNUmakefile.llvm:223: clang option -flto is not working - maybe LLVMgold.so not found - cannot enable LTO mode
In file included from /nix/store/alhk4bwig4mh1qqw0h7y6gkxa64kma84-glibc-2.33-56-dev/include/bits/libc-header-start.h:33,
from /nix/store/alhk4bwig4mh1qqw0h7y6gkxa64kma84-glibc-2.33-56-dev/include/stdlib.h:25,
from src/afl-common.c:26:
/nix/store/alhk4bwig4mh1qqw0h7y6gkxa64kma84-glibc-2.33-56-dev/include/features.h:397:4: error: #warning _FORTIFY_SOURCE requires compiling with optimization (-O) [-Werror=cpp]
397 | # warning _FORTIFY_SOURCE requires compiling with optimization (-O)
| ^~~~~~~
cc1: all warnings being treated as errors
make: *** [GNUmakefile:423: src/afl-common.o] Error 1
thread 'main' panicked at 'assertion failed: status.success()', /home/kvark/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.11.1/build.rs:32:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: failed to compile `afl v0.11.1`, intermediate artifacts can be found at `/run/user/1000/cargo-install0HIprX`Hi, @kvark. Thanks for the detailed output.
It looks like AFLplusplus is failing to build.
The part that says llvm-config is not helping us gets my attention.
Do you have the llvm tools installed? In particular, do you have an llvm-config-13 in your PATH? (I think current stable Rust requires LLVM 13.)
Hmm, I don't have llvm-config-13, but I do have llvm-config:
[nix-shell:/x/code]$ llvm-config --version
13.0.0
Here is one place where things seem to be going wrong: https://github.com/AFLplusplus/AFLplusplus/blob/9321a24e682b5c8bf6278961bd014cb883b87295/GNUmakefile.llvm#L120
Could you please share the output of the following commands:
llvm-config --bindir
ls `llvm-config --bindir`
If clang is not in the listing produced by the latter, could you try installing it, and then try installing afl.rs again?
That question lead me to the rabbit hole of LLVM and clang configuration on Nix... Sorry to be that user on a strange system!
Found https://discourse.nixos.org/t/how-to-correctly-populate-a-clang-and-llvm-development-environment-using-nix-shell/3864, which is fixed in NixOS/nixpkgs#85489, which got... stalled.
I'm also seeing that my configuration has LIBCLANG_PATH:
[nix-shell:/x/code]$ echo $LIBCLANG_PATH
/nix/store/ny2bcqi3ldjqj0qkk2ry7a61jzx5rm3v-clang-13.0.0-lib/lib
Some other projects facing this issue and solving it by different means, like jacobdufault/cquery#237
At this point, I don't know if it's a purely NixOS issue, or also something to be improved in the AFL build process.
I'd love to have some sort of a workaround.
It should be noted that the LLVM-related messages are non-fatal diagnostics, and do not necessarily indicate that LLVM/libclang won't link just fine; the actual fatal error here is due to something unrelated hitting -Werror, which should not be present on build scripts intended used for general distribution for this reason.
I'd love to have some sort of a workaround.
I don't know a lot about Nix. Could you just put a symlink in place to point to the real clang? E.g.,
ln -s `which clang` `llvm-config --bindir`/clang
Thanks for your comment, @Ralith. I understand your point that -Werror should not be used in build scripts. But this appears to be in AFLplusplus's build script. So I think we're kind of stuck with it.
The issue could be pursued upstream, and a patched version used in the mean time.
Actually, the -Werror seems to be coming from here: https://github.com/AFLplusplus/AFLplusplus/blob/9321a24e682b5c8bf6278961bd014cb883b87295/GNUmakefile#L139
@kvark Is it possible you have DEBUG=1 in your environment when you are running cargo install?
No, echo $DEBUG says nothing. My shell configuration is here in case it's useful.
I tried cloning https://github.com/AFLplusplus/AFLplusplus and just building it with make under the same shell. Some warnings are spewed, but I'm not seeing "-Werror" stopping them. It fails with this:
[-] PROGRAM ABORT : Oops, failed to execute '/nix/store/pm454wwwcpa2prhk8qf8s0icbsj2fbxq-llvm-13.0.0/bin/clang' - check your PATH
Location : main(), src/afl-cc.c:2146
ln -s
which clangllvm-config --bindir/clang
Sorry, I can't do this:
ln: failed to create symbolic link '/nix/store/pm454wwwcpa2prhk8qf8s0icbsj2fbxq-llvm-13.0.0/bin/clang': Read-only file system
Also note that clang is not found by the build regardless of whether I enable clang or not in the shell configuration with this:
nix-shell -p llvmPackages_latest.clang
Sounds like it's trying to bake in the path to the LLVM build and assumes that'll be a suitable prefix for clang, which it isn't. If it just executed clang and let PATH do its magic--or alternatively, baked in a path extracted by scanning PATH at buildtime--this wouldn't be an issue.
I tried cloning https://github.com/AFLplusplus/AFLplusplus and just building it with
makeunder the same shell. Some warnings are spewed, but I'm not seeing "-Werror" stopping them. It fails with this:[-] PROGRAM ABORT : Oops, failed to execute '/nix/store/pm454wwwcpa2prhk8qf8s0icbsj2fbxq-llvm-13.0.0/bin/clang' - check your PATH
Location : main(), src/afl-cc.c:2146
Could share the log up to that point? Also, afl.rs currently uses commit 9321a24. Could you try with that commit specifically?
Looks relevant to AFLplusplus/AFLplusplus#316, which is claimed to be fixed.
Here is the full log on that revision with gmp package enabled: make.log
https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/security/aflplusplus/default.nix may also be of interest
I've sadly run into this as well on NixOS. Has anyone maybe found a working solution in the meantime?
The problem is that the AFLplusplus makefiles only check whether DEBUG is set or not. But cargo always sets DEBUG to true or false and so convinces AFLplusplus to build in debug mode with -Werror set. I've made PR #248 with a minimalistic patch to build.rs.
The clang and llvm issues seem to be entirely separate. To get a clang build on NixOS I put some dirty hacks into https://github.com/vkleen/zorn/blob/3e18f4444c9390eda6b2e0ec5e23285ecfac0a6e/flake.nix
Essentially, I'm just merging llvm, clang and lld into a single symlinked path hierarchy. Use at your own risk (I'm only hacking this together for development).