l4-bender usage with Rust

[sevenautumns]

Hello,
I am currently trying to extract the Rust libraries from this repository for making compilation of binaries for l4re-targets a little bit faster.
Right now, for compiling Rust projects, it is required to build everything of l4re. And this takes quite some time.
So I extracted the libraries for (maybe) compiling l4re-binaries of rust programs directly.
But the linker part of the build seems to hinder this endeavour .
When cargo build finishes the build by calling in the linker, there are a few problems.
(I'll paste the l4-bender command, which is invoked by cargo below)

• The first problem seems to be that all of these paths are interpreted as if they were relative paths, so I had to call this command from the root-directory
• The second following problem seems to be that every single file (and folder) which is mentioned in the command, is interpreted as a .pc-file. Even the last part of the command with -o which should normally define the output file (but here it is used as an input with a .pc ending?)

For compiling I used my "own" rust toolchain:
https://github.com/Steav005/rust/tree/update-l4re-target
Which is based upon atopia's rust toolchain
https://github.com/atopia/rust/tree/update-l4re-target
Which is based upon the recommended toolchain
https://github.com/humenda/rust

For l4-bender, I used the latest snapshot version 21.07

The main problem seems to be that the called l4-bender command is wrong.
With this custom rust toolchain and the 21.07 l4-bender, basic compilation of this repository works flawlessly for the amd64 target. (And I have never seen a .pc-file in this repo's outputs).
I know that this repository uses make and rustc directly for compiling rust libraries/binaries.
But there are to many files to this project for me to grasp the entire compilation process.

How should the correct l4-bender-command be structured ?
Is it even possible to build binaries for l4re externally while static-linking the required libraries?
Are there documents on how to use l4-bender?

The failing l4-bender invocation

"l4-bender" "--" "/home/autumnal/l4rust/target/x86_64-unknown-l4re-uclibc/release/examples/hello-ab8330dafea89441.hello.5qhzz9zp-cgu.0.rcgu.o" "/home/autumnal/l4rust/target/x86_64-unknown-l4re-uclibc/release/examples/hello-ab8330dafea89441.hello.5qhzz9zp-cgu.1.rcgu.o" "/home/autumnal/l4rust/target/x86_64-unknown-l4re-uclibc/release/examples/hello-ab8330dafea89441.hello.5qhzz9zp-cgu.2.rcgu.o" "/home/autumnal/l4rust/target/x86_64-unknown-l4re-uclibc/release/examples/hello-ab8330dafea89441.45jtgf841qjk2mhz.rcgu.o" "-L" "/home/autumnal/l4rust/target/x86_64-unknown-l4re-uclibc/release/deps" "-L" "/home/autumnal/l4rust/target/release/deps" "-L" "/run/media/autumnal/ssddata/GitRepos/rust/patched/lib/rustlib/x86_64-unknown-l4re-uclibc/lib" "--start-group" "-static" "/run/media/autumnal/ssddata/GitRepos/rust/patched/lib/rustlib/x86_64-unknown-l4re-uclibc/lib/libstd-3603de956c965eaf.rlib" "/run/media/autumnal/ssddata/GitRepos/rust/patched/lib/rustlib/x86_64-unknown-l4re-uclibc/lib/libpanic_abort-601cd4d89ed16e0c.rlib" "/run/media/autumnal/ssddata/GitRepos/rust/patched/lib/rustlib/x86_64-unknown-l4re-uclibc/lib/liblibc-5b45eb57eb334c25.rlib" "/run/media/autumnal/ssddata/GitRepos/rust/patched/lib/rustlib/x86_64-unknown-l4re-uclibc/lib/libstd_detect-12b236156e9d4b91.rlib" "/run/media/autumnal/ssddata/GitRepos/rust/patched/lib/rustlib/x86_64-unknown-l4re-uclibc/lib/librustc_demangle-e991672d04416a27.rlib" "/run/media/autumnal/ssddata/GitRepos/rust/patched/lib/rustlib/x86_64-unknown-l4re-uclibc/lib/libhashbrown-819e04a2f2f36d10.rlib" "/run/media/autumnal/ssddata/GitRepos/rust/patched/lib/rustlib/x86_64-unknown-l4re-uclibc/lib/librustc_std_workspace_alloc-acfa987dd0f2640c.rlib" "/run/media/autumnal/ssddata/GitRepos/rust/patched/lib/rustlib/x86_64-unknown-l4re-uclibc/lib/libunwind-c4492c0f6288c7d7.rlib" "/run/media/autumnal/ssddata/GitRepos/rust/patched/lib/rustlib/x86_64-unknown-l4re-uclibc/lib/libcfg_if-283b1de975de9b2c.rlib" "/run/media/autumnal/ssddata/GitRepos/rust/patched/lib/rustlib/x86_64-unknown-l4re-uclibc/lib/liblibc-1c300f9078f18581.rlib" "/run/media/autumnal/ssddata/GitRepos/rust/patched/lib/rustlib/x86_64-unknown-l4re-uclibc/lib/liballoc-77abbfbc930f2f26.rlib" "/run/media/autumnal/ssddata/GitRepos/rust/patched/lib/rustlib/x86_64-unknown-l4re-uclibc/lib/librustc_std_workspace_core-652b8db2aa5aec18.rlib" "/run/media/autumnal/ssddata/GitRepos/rust/patched/lib/rustlib/x86_64-unknown-l4re-uclibc/lib/libcore-22323b437be4c831.rlib" "--end-group" "/run/media/autumnal/ssddata/GitRepos/rust/patched/lib/rustlib/x86_64-unknown-l4re-uclibc/lib/libcompiler_builtins-63dc035dd5f6e148.rlib" "-L" "/run/media/autumnal/ssddata/GitRepos/rust/patched/lib/rustlib/x86_64-unknown-l4re-uclibc/lib" "-o" "/home/autumnal/l4rust/target/x86_64-unknown-l4re-uclibc/release/examples/hello-ab8330dafea89441" "--gc-sections" "-O2" "-nostdlib"

[humenda]

Hi

I am currently trying to extract the Rust libraries from this repository for making compilation of binaries for l4re-targets a little bit faster. Right now, for compiling Rust projects, it is required to build everything of l4re. And this takes quite some time.

AFAIK, this is not the case. You need to build everything once. Later, you can go to the corresponding subdirectory and invoke make there and it'll built just this again.

So I extracted the libraries for (maybe) compiling l4re-binaries of rust programs directly. But the linker part of the build seems to hinder this endeavour . When `cargo build` finishes the build by calling in the linker, there are a few problems. (I'll paste the l4-bender command, which is invoked by cargo below) - The first problem seems to be that all of these paths are interpreted as if they were relative paths, so I had to call this command from the root-directory - The second following problem seems to be that every single file (and folder) which is mentioned in the command, is interpreted as a `.pc`-file. Even the last part of the command with `-o` which should normally define the output file (but here it is used as an input with a .pc ending?) For compiling I used my "own" rust toolchain: https://github.com/Steav005/rust/tree/update-l4re-target Which is based upon atopia's rust toolchain https://github.com/atopia/rust/tree/update-l4re-target Which is based upon the recommended toolchain https://github.com/humenda/rust

I wasn't aware that this has been modified in the mantime. I never had the time to properly blog about this, so here's a quick summary about why things are as they are. Rust/Cargo insist on calling the linker (except for work-arounds, see further below). The same goes for BID, the L4Re build system. L4Re relies on quite some linker magic and hence the make logic was at some point all put into a large Perl script called l4-bender. L4-bender calls the linker and bends the linker arguments passed to it further to pass it to ld. The issue with l4-bender is that it behaves almost like ld, but really just almost. E.g. the order of arguments matters, especially the separation of "--". Rustc has only limited support of passing through linker arguments and in particular, the order is hard to influence. In addition, l4-bender needs arguments passed from Make that contains spaces and these cannot be passed to Rustc. My solution for this has been to add a new Rust linker variant l4-bender. This was actually suggested by some Rust folks. The issue is that the linker backend changes frequently and there has been no automatic testing of it. L4Re uses PC files to find its libraries in the build tree and l4-bender unwraps the PC files and passes the required arguments to ld. I did not look at the links that you've provided, but it is very likely that the L4Bender struct in the compiler has a bug so that it assumes to make more things a PC-file-argument where it shouldn't be.

The main problem seems to be that the called `l4-bender` command is wrong. With this custom rust toolchain and the `21.07 l4-bender`, basic compilation of this repository works flawlessly for the `amd64` target. (And I have never seen a `.pc`-file in this repo's outputs).

Under `obj/**/pc`?

I know that this repository uses `make` and `rustc` directly for compiling rust libraries/binaries. But there are to many files to this project for me to grasp the entire compilation process.

Yeah, the relevant bits are in src/l4/mk. The rest are mostly Makefiles including other makefiles or those in obj/ that are auto-generated. It's all a bit arcane.

How should the correct `l4-bender`-command be structured ? Is it even possible to build binaries for l4re externally while static-linking the required libraries? Are there documents on how to use `l4-bender`?

2): yes. 1) 3): L4Re lacks documentation on the build system, apart from the initial README. The only docs that you can get are the undocumented sources themselves. Given the current small interest in maintaining the l4-bender linker variant in Rustc, I would suggest the following: 1. Building of libraries stays the same (lib.mk). 2. Applications are built as static libraries. - This needs to be specified in Cargo.toml for each application. - `main.rs` has to export a C-compatible main function. - in bin.mk, binary.inc and maybe rules.mk, the rules that contain `RS` and `RUST` need to be adapted. They would need to be a copy of the C versions, though not linking object files, but the static archive that was produced. I don't know whether you have time and energy to look into this and whether my description has been clear enough. I have tried to update my Rust toolchain to a recent Rustc, but the L4Bender support is completely broken and I would basically need to rewrite the l4-bender support. I lack the time to do so. This is also the reason why I did not do anything about the other PRs of yours: I cannot test them. The steps to change to static library building could be done in around 1-3 weeks and I'd be happy to help out in case you would like to look into it. And even though time is a limited resource on my side, I'm definitely around for helping to investigate issues. Please let me know what you think.

[sevenautumns]

1. Building of libraries stays the same (lib.mk).

I agree with this.

AFAIK, this is not the case. You need to build everything once. Later, you can
go to the corresponding subdirectory and invoke make there and it'll built
just this again.
If it is only required to run the make command

If this is the case, there is no apparent need to move the rust libraries outside.

I have tried to update my Rust toolchain to
a recent Rustc, but the L4Bender support is completely broken and I would
basically need to rewrite the l4-bender support.

I did not notice this at all. In fact, I was able to use the newest stable toolchain, which was released just 11 days ago.

This is also the reason why I did not do anything about the other PRs of
yours: I cannot test them.

Please try using my 1.55.0 toolchain for l4re:
https://github.com/Steav005/rust/tree/update-l4re-target-1.55.0
I have replaced the official Rust-Readme with build instructions on my version.
Both targets (x86_64 and armv7) are supported.
For compiling my arm-support branch I used the newest arm-none-linux-gnueabihf-gcc

With his brand-new toolchain supported, we could think about a pull request from my toolchain into yours should it work for you

[humenda]

> I have tried to update my Rust toolchain to a recent Rustc, but the L4Bender support is completely broken and I would basically need to rewrite the l4-bender support. I did not notice this at all. In fact, I was able to use the newest stable toolchain, which was released just 11 days ago. > This is also the reason why I did not do anything about the other PRs of yours: I cannot test them. Please try using my 1.55.0 toolchain for l4re: https://github.com/Steav005/rust/tree/update-l4re-target-1.55.0 I have replaced the official Rust-Readme with build instructions on my version. Both targets (x86_64 and armv7) are supported.

I will do so, but I unfortunately have to postpone this again to the beginning of October, since a rather tight deadline is approaching. Looking into this is the next high priority, though. Thanks

[wucke13]

Just a small comment: I would much prefer if one can build code for L4RE without having to compile any L4RE stuff. The L4RE build process is kind of clunky, and from a roles perspective it's preferable when an application engineer doesn't need to have the integration engineers knowledge to build an application. Building a static lib from cargo, which then is just copied & linked into L4RE should be possible, right?

[humenda]

@sevenautumns I have resumed work on L4Rust. Would you mind checking your issue
with latest rustc? @hargoniX patched L4-bender support so that it works with
L4Re again. I'm currently cleaning up the code base and am awaiting patches from
@hargoniX.

[hargoniX]

There is an additional patch by atopia necessary at the bare minimum, I cherry picked it to my fork. It is this one: atopia@f769f1f

Have you correctly rebased master by now? I'll check it out on monday.

[humenda]

Have you correctly rebased master by now? I'll check it out on monday.

I'll ping you off-line.