fails to build on Debian 10
victorjulien opened this issue · 22 comments
I suspect this is a Debian issue, but not sure. Recently Debian 10 updated its rustc to 1.41.1 (and cargo to 1.42.1). Since then this crate fails to build inside Suricata:
error: /builds/inliniac/suricata-ci/suricata/rust/target/release/deps/libder_oid_macro-6303e17a207c2efa.so: undefined symbol: llvm.x86.subborrow.64
--> /builds/inliniac/suricata-ci/suricata/rust/vendor/der-parser/src/lib.rs:171:9
|
171 | pub use der_oid_macro::oid;
| ^^^^^^^^^^^^^
This happens in Suricata CI builds.
Interestingly it does not happen on Debian 10 on ARM. Not sure what could cause this.
Is it using everything from cargo, or debian packages?
I've already heard of this error, but I have no diagnosis for the moment. It seems related to the rust compiler / LLVM toolchain versions, I'm not sure where the bug is.
It installs rust deps from cargo.
Ok, I was able to reproduce the problem on a fresh Debian 10 install.
I also suspect a bug in the Debian version of cargo/rustc. On the same host:
- if using the provided compiler, I have the same error
- if using rustup 1.47 version, build is OK
- if using rustup 1.41.1, build is also OK
That said, I do not have yet a solution
Playing a bit more with this. ARM on Debian 10 does not have the issue. It compiles fine.
# docker run -it debian:10 /bin/bash
apt update && apt upgrade -y && apt install -y cargo rustc git
git clone https://github.com/rusticata/der-parser -b der-parser-4.x
cd der-parser/
cargo build
Compiling der-parser v4.1.0 (/der-parser)
warning: unknown lint: `broken_intra_doc_links`
--> src/lib.rs:142:9
|
142 | #![deny(broken_intra_doc_links)]
| ^^^^^^^^^^^^^^^^^^^^^^
|
= note: `#[warn(unknown_lints)]` on by default
Finished dev [unoptimized + debuginfo] target(s) in 1m 23s
Also tried a 32 bit x86. It fails with a slightly different but probably related issue
Debian 9 on x86_64 also has the issue. I did not try that on other archs.
arm64 works as well. So in the archs I have access to, x86_64 and 32 fail, arm and arm64 work.
Do we want to file a bug for rustc in stable?
This is interesting because for me, I can build a Suricata 6.0.0 backport fine in a buster cowbuilder chroot with
rustc (1.41.1+dfsg1-1~deb10u1)libstd-rust-dev:amd64 (1.41.1+dfsg1-1~deb10u1)libstd-rust-1.41:amd64 (1.41.1+dfsg1-1~deb10u1)llvm (1:7.0-47)
Making all in rust
make[3]: Entering directory '/build/suricata-6.0.0/rust'
\
CARGO_HOME="/build/suricata-6.0.0/debian/cargohome" \
CARGO_TARGET_DIR="/build/suricata-6.0.0/rust/target" \
/usr/bin/cargo build --release \
--features "function-macro lua "
Compiling autocfg v1.0.1
Compiling cfg-if v0.1.9
Compiling semver-parser v0.7.0
Compiling arrayvec v0.4.12
Compiling ryu v1.0.5
Compiling bitflags v1.2.1
Compiling libc v0.2.79
Compiling nodrop v0.1.14
Compiling version_check v0.9.2
Compiling memchr v2.3.3
Compiling static_assertions v0.3.4
Compiling getrandom v0.1.15
Compiling siphasher v0.3.3
Compiling ppv-lite86 v0.2.9
Compiling proc-macro2 v0.4.30
Compiling proc-macro-hack v0.5.18
Compiling unicode-xid v0.1.0
Compiling syn v0.15.44
Compiling build_const v0.2.1
Compiling num-derive v0.2.5
Compiling byteorder v1.3.4
Compiling base64 v0.11.0
Compiling uuid v0.8.1
Compiling widestring v0.4.3
Compiling md5 v0.7.0
Compiling semver v0.9.0
Compiling phf_shared v0.8.0
Compiling crc v1.8.1
Compiling num-traits v0.2.12
Compiling num-integer v0.1.43
Compiling num-bigint v0.2.6
Compiling num-bigint v0.3.0
Compiling num-rational v0.2.4
Compiling num-iter v0.1.41
Compiling num-complex v0.2.4
Compiling nom v5.1.1
Compiling phf v0.8.0
Compiling rustc_version v0.2.3
Compiling quote v0.6.13
Compiling time v0.1.44
Compiling rand_core v0.5.1
Compiling rand_chacha v0.2.2
Compiling rand_pcg v0.2.1
Compiling lexical-core v0.6.7
Compiling num-traits v0.1.43
Compiling rand v0.7.3
Compiling enum_primitive v0.1.1
Compiling phf_generator v0.8.0
Compiling phf_codegen v0.8.0
Compiling num v0.2.1
Compiling tls-parser v0.9.3
Compiling rusticata-macros v2.1.0
Compiling ntp-parser v0.4.0
Compiling der-oid-macro v0.2.0
Compiling der-parser v3.0.4
Compiling ipsec-parser v0.5.0
Compiling x509-parser v0.6.5
Compiling der-parser v4.1.0
Compiling kerberos-parser v0.5.0
Compiling snmp-parser v0.6.0
Compiling suricata v6.0.0 (/build/suricata-6.0.0/rust)
Finished release [optimized + debuginfo] target(s) in 1m 21s
but that fails with:
Compiling der-parser v4.1.0
error: /build/suricata-6.0.1/rust/target/release/deps/libder_oid_macro-6303e17a207c2efa.so: undefined symbol: llvm.x86.subborrow.64
--> /build/suricata-6.0.1/rust/vendor/der-parser/src/lib.rs:171:9
|
171 | pub use der_oid_macro::oid;
| ^^^^^^^^^^^^^
error: aborting due to previous error
error: could not compile `der-parser`.
for 6.0.1 although, according to git, no code changes were made in the vendored der-parser code.
Also, buster surely didn't update versions, not in stable! There was the following update to 1.41.1 in September:
rustc (1.41.1+dfsg1-1~deb10u1) buster; urgency=medium
* Non-maintainer upload.
* Backport to buster.
* stage0 build.
- Use arm-unknown-linux-gnueabi target for armel.
* Use LLVM 7.
* Disable wasm.
* Reduce debugging symbols on i386 to avoid FTBFS due to OOM.
-- Emilio Pozuelo Monfort <pochu@debian.org> Tue, 08 Sep 2020 18:38:19 +0200
Maybe the switch to LLVM 7 was involved?
6.0.0 compiled file for me in Debain 10 (x86_64) 0 6.0.1 doesn't, same as above. I imagine der-parser wasn't part of rust in that release?
6.0.0 compiled file for me in Debain 10 (x86_64) 0 6.0.1 doesn't, same as above. I imagine der-parser wasn't part of rust in that release?
Both 6.0.0 and 6.0.1 ship the same vendored der-parser source,
6.0.0 compiled file for me in Debain 10 (x86_64) 0 6.0.1 doesn't, same as above. I imagine der-parser wasn't part of rust in that release?
Both 6.0.0 and 6.0.1 ship the same vendored der-parser source,
Okay, I see that now on another make of it. Compiles fine 6.0.0. That said, would this then be indicative of a suricata 6.0.1 problem?
Update: Tried Ubuntu LTS, 6.0.1, compiles fine. As mentioned, it probably is Debian related.
If you compile Suricata from git, both the 6.0.0 tag and 6.0.1 tag fail. So maybe it is related to the vendored rust crates in the release tarballs for 6.0.0 and 6.0.1
I can get the der-parser-4.x branch to compile by simply commenting out the der-oid-macro crate:
diff --git a/src/lib.rs b/src/lib.rs
index 78b1e86..5d10a80 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -168,6 +168,6 @@ pub use rusticata_macros::{custom_check, flat_take};
use proc_macro_hack::proc_macro_hack;
-#[proc_macro_hack]
-/// Procedural macro to get encoded oids, see the [oid module](oid/index.html).
-pub use der_oid_macro::oid;
+//#[proc_macro_hack]
+// Procedural macro to get encoded oids, see the [oid module](oid/index.html).
+//pub use der_oid_macro::oid;I don't know what effects this would have, but maybe we can work around the issue this way?
The side-effect is that crates using der-parser (like x509-parser will probably fail to build), but other than dependencies, this has no side effect on der-parser itself.
OTOH, this also seems to confirm this is related to LLVM, and the 2 crates (der-oid-macro and der-parser) possibly linked with different versions. That can happen in distros, when crates are built at different times.
The macro is a re-export from der-oid-macro::oid, so it is not absolutely necessary to re-export it.
I'll try to see if the macro can be removed from der-parser, without breaking all other crates and other side-effect.
Hum, github closed by because of commit message of the possible fix. Until confirmation that bug is really fixed, I'm reopening this
Can you suggest how to test this?
If you compile Suricata from git, both the 6.0.0 tag and 6.0.1 tag fail. So maybe it is related to the vendored rust crates in the release tarballs for 6.0.0 and 6.0.1
Suricata still fails to build due to this error as of release 6.0.9.
Compiling der-parser v4.1.0
error: /opt/suricata-6.0.9/rust/target/release/deps/libder_oid_macro-5c3e723d2ee71310.so: undefined symbol: llvm.x86.subborrow.64
--> /opt/suricata-6.0.9/rust/vendor/der-parser/src/lib.rs:171:9
|
171 | pub use der_oid_macro::oid;
| ^^^^^^^^^^^^^
error: aborting due to previous error
error: could not compile `der-parser`.
warning: build failed, waiting for other jobs to finish...
error: build failed
make[1]: *** [Makefile:548: all-local] Error 101
make[1]: Leaving directory '/opt/suricata-6.0.9/rust'
make: *** [Makefile:492: all-recursive] Error 1
@chifflier this continues to affect Suricata 6 (current stable) on Debian 10. Can we get some kind of workaround? In Debian packaging they're working around it by commenting out 2 lines in src/lib.rs: https://redmine.openinfosecfoundation.org/issues/4204#note-4
Would be great if we could get a 4.1.1 with this work around, perhaps behind a flag of sorts.
Hi @victorjulien
Version 4.1 is quite old :/
For version 4.1, I think I can add a feature to enable/disable reexport of the macro. It would be enabled by default, so to disable it, suricata will have to overrides the features in its Cargo.toml file, using something like:
der-parser = { version=4.1.1, default-features = false }
This should work for version 4.x, but I don't know if a similar workaround will be required for later version of der-parser.
That would be great.
Our upcoming Suricata 7 release uses 6.0.1 directly and also has 8.2 as an indirect dependency. We haven't seen similar issues there, but it should be noted that we've bumped our MSRV so we're not looking at Debian 10's rust version anymore.
On the topic of old versions, I feel there is a mismatch between the goals of rust "systems programming" and much of the ecosystem "devops" attitude. "System programs" are often running for a long time, with strict stability requirements and minimal updates. Suricata 6 is a few years out by now, and was in development a few years, so that is a quite some time, but nothing extraordinary I think. This is not a complaint about your efforts here, but just an observation about the larger rust effort.
So I just pushed 4d566f9 which allows disabling the proc-macro causing problems.
Hopefully this should fix the problem, but I guess there is no easy way to test before a new release is out, so I'll release 4.1.1 and we'll see it that fixes it.
Important note: the workaround is not included in other versions or branches. If we have the problem on another version, please reopen another issue and I'll include in master and for all supported versions.
I fully agree on the mismatch between "stability" and "devops": most common projects consider that everyone is using the latest compiler/dependencies/os/... and it's becoming very hard to maintain stable versions in many cases. It's not specific to Rust, but unfortunately very common.
Version 4.1.1 has be released, so I'm closing this issue. Feel free to reopen or open another issue if the problem is still there.
Thanks a lot @chifflier!