/rust_omnibus

A prototypical approach to synthesizing a single rust static lib from arbitrarily many crates, for linking into a C++ project.

Primary LanguageShell

Summary

This project demonstrates a technique for synthesizing a "omnibus" static library from arbitrarily many Rust crates, to mitigate the fact that it is not possible to link multiple Rust-static-libs into a single C(++) project.

Usage

Navigate to the client directory and run make. It expects the following dependencies to be installed:

  1. cargo
  2. cbindgen
  3. jq

Discussion

C++ codebases that wish to incrementally introduce Rust must do so by compiling the Rust sources into a static library ("staticlib" in Cargo.toml), which exposes a C-ABI-compatible interface (by marking the function as #[no_mangle]). Tools such as cbindgen and cxx.rs help by generating the headers and/or glue code necessary for C/C++ code to call into the Rust-built static library.

Naturally, codebases that wish to introduce Rust will want to do so incrementally and asynchronously, in multiple components of their project (say, the file parser and the virtual memory system). In doing so, they'll discover a limitation of the language: Rust does not support linking multiple rust-generated static libraries together (see also rust #44322). Apparently, the language/build system provides no way to hide the standard library symbols, so each static library would bring along its own copy, causing (at best) linker errors or (at worst) ODR violations.

Even if Rust provided a method to hide standard library symbols, name collisions are still a risk. Since the symbols at the FFI boundary are not mangled, if two or more static libraries expose the same symbol name, there is no guarantee which one will be chosen by the linker.

Taken together, it would seem a single application can depend on at most one rust-built static library. It would be impractical and unwise to do all Rust development (for entirely distinct components) in a single project. Instead, we would like to develop small Rust crates in appropriate subdirectories of our project, and let the aforementioned limitations be handled invisibly by our build system. This repository attempts to facilitate such a solution.

Output

out/omni contains all of the artifacts generated by make.

out/omni/include contains one header per Rust module, generated by cbindgen. It has the structure:

client/out/omni/include
└── rust
    ├── liba
    │   └── bindings.h
    └── libb
        └── bindings.h

Notice that while the aforementioned language limitations require we generate a single static library, we can produce separate headers to minimize the overhead of incremental compilation.

out/omni/client is a regular executable.

Considerations

  1. liba and libb both depend on (different versions of) the rand crate. The output of cargo tree can show us how those dependencies were reconciled (<<< emphasis added, local file paths removed):
omni v0.1.0
├── liba v0.1.0
│   └── rand v0.7.3 <<<
│       ├── getrandom v0.1.16
│       │   ├── cfg-if v1.0.0
│       │   └── libc v0.2.155
│       ├── libc v0.2.155
│       ├── rand_chacha v0.2.2
│       │   ├── ppv-lite86 v0.2.17
│       │   └── rand_core v0.5.1
│       │       └── getrandom v0.1.16 (*)
│       └── rand_core v0.5.1 (*)
└── libb v0.1.0
    └── rand v0.8.5 <<<
        ├── libc v0.2.155
        ├── rand_chacha v0.3.1
        │   ├── ppv-lite86 v0.2.17
        │   └── rand_core v0.6.4
        │       └── getrandom v0.2.15
        │           ├── cfg-if v1.0.0
        │           └── libc v0.2.155
        └── rand_core v0.6.4 (*)

Here, we see that the resolver selected both rand versions, but was able to deduplicate some of rand's dependencies (getrandom and rand_core, indicated with (*)). However, both versions of rand (0.7.3 and 0.8.5) are linked into the final executable (modulo unused symbol stripping).

  1. liba and libb depend on different versions of the Rust language itself, with the rust-version field in their respective Cargo.toml files. Note that these fields specify the minimum Rust version required; it is not possible to specify a maximum version required.

To approximate a real-world use-case for this, liba uses a relatively new Rust language feature: Option::inspect, and therefore requires Rust 1.76. Similarly, libb uses std::thread::available_parallelism, introduced in Rust 1.59.

  1. libb has a dependency, autocfg, which depends on a very old Rust edition. Editions are backwards-incompatibile language versions. Note that crates compiled in one edition operate seamlessly with those compiled in other editions.