How to link rust-mbedtls to a no_std project?

Question

How to link rust-mbedtls to a no_std project?

yaoxin1995 opened this issue 2 years ago · 10 comments

yaoxin1995 commented 2 years ago

Hey,

After reading the doc, I am still pretty confused about how to add this crate to a no_std project.

Could someone provide me with some hints?

Besides adding mbedtls = "0.8.1" to Cargo.toml, what should I do?

And where is the build script, sorry I didn't find any script call "build*.sh"

Sorry I am a beginner in rust and I am super confused

Best,
Yaoxin

Answer 1 · 2023-02-21T03:31:29.000Z

Maybe you can try mbedtls = { version = "0.8.1", default-features = false, features = ["no_std_deps"] }

Answer 2 · 2023-02-21T16:10:23.000Z

Maybe you can try mbedtls = { version = "0.8.1", default-features = false, features = ["no_std_deps"] }

@weibin159 Many thanks,

But it seems that mbedtls = { version = "0.8.1", default-features = false, features = ["no_std_deps"] } depends on a core_io crate that require std libs:

error: failed to run custom build command for `core_io v0.1.20210325`

Caused by:
  process didn't exit successfully: `/home/yaoxin/master-thesis-quark/qkernel/../target/release/build/core_io-e6fa2f26978f4df5/build-script-build` (exit status: 101)
  --- stderr
  thread 'main' panicked at 'Unknown compiler version, upgrade core_io?', /home/yaoxin/.cargo/registry/src/github.com-1ecc6299db9ec823/core_io-0.1.20210325/build.rs:73:69
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error[E0463]: can't find crate for `std`
  |
  = note: the `x86_64-qkernel-13220628803078393756` target may not support the standard library
  = note: `std` is required by `once_cell` because it does not declare `#![no_std]`
  = help: consider building the standard library from source with `cargo build -Zbuild-std`

error[E0463]: can't find crate for `std`
  |
  = note: the `x86_64-qkernel-13220628803078393756` target may not support the standard library
  = note: `std` is required by `byteorder` because it does not declare `#![no_std]`
  = help: consider building the standard library from source with `cargo build -Zbuild-std`

error[E0463]: can't find crate for `std`
 --> /home/yaoxin/.cargo/registry/src/github.com-1ecc6299db9ec823/byteorder-1.4.3/src/io.rs:1:5
  |
1 | use std::{
  |     ^^^ can't find crate
  |
  = note: the `x86_64-qkernel-13220628803078393756` target may not support the standard library
  = help: consider building the standard library from source with `cargo build -Zbuild-std`

^Cmake[1]: *** [makefile:28: kernel] Interrupt46: byteorder, once_cell                                                                                                                                                                                          
make: *** [makefile:5: release] Interrupt

In my case, I don't have std libs.:

{
  "llvm-target": "x86_64-unknown-none",
  "data-layout": "e-m:e-i64:64-f80:128-n8:16:32:64-S128",
  "linker-flavor": "gcc",
  "target-endian": "little",
  "target-pointer-width": "64",
  "target-c-int-width": "32",
  "arch": "x86_64",
  "os": "none",
  "disable-redzone": true,
  "features": "-mmx,-sse,-avx,+soft-float",
  "panic-strategy": "abort"
}

Do you have any idea how should I resolve the issue?

Answer 3 · 2023-03-10T06:59:16.000Z

The core_io dependency was removed in #211. If your context allows, you can point to a specific commit until we publish a new crate version.

Answer 4 · 2023-03-11T11:17:59.000Z

The core_io dependency was removed in #211. If your context allows, you can point to a specific commit until we publish a new crate version.
@zugzwang @DrTobe

Thanks for the reply and fix the core_io issue.

As [zugzwang](https://github.com/zugzwang) suggested, I point to the commit that removes the core_io. But I got another error when I compile the project. Could you take a look and give me some suggestions?

How I link the mbedtls:

mbedtls = {git = "https://github.com/fortanix/rust-mbedtls.git", rev = "2879a2a54b503a653c2337b81b519c5905f23ff4", default-features = false, features = ["no_std_deps"] }

the compilation configuration:

{
  "llvm-target": "x86_64-unknown-none",
  "data-layout": "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128",
  "linker-flavor": "gcc",
  "target-endian": "little",
  "target-pointer-width": "64",
  "target-c-int-width": "32",
  "arch": "x86_64",
  "os": "none",
  "disable-redzone": true,
  "features": "-mmx,-avx,-sse,+soft-float",
  "frame-pointer": "always"
}

the error I got:

RUSTFLAGS="--cfg aes_force_soft --cfg polyval_force_soft" CARGO_TARGET_DIR=../target cargo xbuild --target x86_64-qkernel.json --release
   Compiling version_check v0.9.4
   Compiling proc-macro2 v1.0.51
   Compiling typenum v1.16.0
   Compiling unicode-ident v1.0.6
   Compiling memchr v2.5.0
   Compiling cfg-if v1.0.0
   Compiling regex-syntax v0.6.28
   Compiling spin v0.5.2
   Compiling humantime v2.1.0
   Compiling semver v1.0.16
   Compiling strsim v0.8.0
   Compiling vec_map v0.8.2
   Compiling bindgen v0.58.1
   Compiling termcolor v1.2.0
   Compiling shlex v1.1.0
   Compiling lazycell v1.3.0
   Compiling autocfg v1.1.0
   Compiling peeking_take_while v0.1.2
   Compiling rustc-hash v1.1.0
   Compiling once_cell v1.17.0
   Compiling hex-literal v0.3.4
   Compiling libloading v0.7.4
   Compiling lazy_static v1.4.0
error[E0463]: can't find crate for `std`
  |
  = note: the `x86_64-qkernel-13220628803078393756` target may not support the standard library
  = note: `std` is required by `once_cell` because it does not declare `#![no_std]`
  = help: consider building the standard library from source with `cargo build -Zbuild-std`

   Compiling log v0.4.17
   Compiling clap v2.34.0
   Compiling clang-sys v1.6.0
   Compiling generic-array v0.14.6
   Compiling nom v5.1.2
   Compiling ahash v0.7.6
   Compiling lock_api v0.4.9
   Compiling rustc_version v0.4.0
   Compiling heapless v0.7.16
   Compiling quote v1.0.23
   Compiling aho-corasick v0.7.20
   Compiling syn v1.0.107
   Compiling spin v0.9.5
error[E0463]: can't find crate for `std`
 --> /home/yaoxin/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.17.0/src/imp_std.rs:6:5
  |
6 | use std::{
  |     ^^^ can't find crate
  |
  = note: the `x86_64-qkernel-13220628803078393756` target may not support the standard library
  = help: consider building the standard library from source with `cargo build -Zbuild-std`

error: cannot find macro `panic` in this scope
   --> /home/yaoxin/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.17.0/src/lib.rs:723:47
    |
723 |                 init.take().unwrap_or_else(|| panic!("Lazy instance has previously been poisoned"))
    |                                               ^^^^^
    |
    = note: consider importing this macro:
            core::panic

There are 2 crate under this repo, one is the mbedtls, another one is mbedtls-sys.

Do I need to link mbedtls-sys to my project at all?

I think that this error comes from mbedtls-sys-auto crate, but how I can disable std for the crate?

   Compiling mbedtls-sys-auto v2.28.0 (https://github.com/fortanix/rust-mbedtls.git?rev=2879a2a54b503a653c2337b81b519c5905f23ff4#2879a2a5)
error[E0463]: can't find crate for `std`
  --> /home/yaoxin/.cargo/registry/src/github.com-1ecc6299db9ec823/bit-vec-0.5.1/src/lib.rs:95:1
   |
95 | extern crate std;
   | ^^^^^^^^^^^^^^^^^ can't find crate
   |
   = note: the `x86_64-qkernel-13220628803078393756` target may not support the standard library
   = help: consider building the standard library from source with `cargo build -Zbuild-std`

error[E0463]: can't find crate for `std`
  |
  = note: the `x86_64-qkernel-13220628803078393756` target may not support the standard library
  = note: `std` is required by `once_cell` because it does not declare `#![no_std]`
  = help: consider building the standard library from source with `cargo build -Zbuild-std`

error[E0463]: can't find crate for `std`

@gaojiaqi7 @MabezDev Sorry to bother you. I saw you guys also tried to use this crate in the no_std environment.

Did you get the error I mentioned when using this crate?

Any idea how to resolve it?

Answer 5 · 2023-03-11T16:03:20.000Z

My question is how to disable the std for crate mbedtls-sys-auto?

Where should I add my own version of the following functions?

[features]
# If you use mbedtls-sys in a no_std configuration, you need to provide your
# own versions of the following functions:
#
# * calloc/free
# * strstr/strlen/strncpy/strncmp/strcmp/snprintf
# * memmove/memcpy/memcmp/memset
# * rand/printf (used only for self tests. optionally use custom_printf)

Will the functions strstr/strlen/strncpy/strncmp/strcmp/snprintf/ memmove/memcpy/memcmp/memset be written in rust or c?

Do you have any reference or example of how to implement these functions and where should I add those functions?

Answer 6 · 2023-03-13T09:55:57.000Z

Do I need to link mbedtls-sys to my project at all?

No, not manually. It is added automatically as a transitive dependency of the mbedtls dependency. If that one is configured with default-features = false, features = ["no_std_deps"], you are good to go (from the Rust side!)

= note: the x86_64-qkernel-13220628803078393756 target may not support the standard library
= note: std is required by once_cell because it does not declare #![no_std]
= help: consider building the standard library from source with cargo build -Zbuild-std

In my dependency graph (cargo tree), I do not see once_cell at all. Are you sure this error is caused by mbedtls? Maybe, you should try with a more minimal example first.

Will the functions strstr/strlen/strncpy/strncmp/strcmp/snprintf/ memmove/memcpy/memcmp/memset be written in rust or c?
Do you have any reference or example of how to implement these functions and where should I add those functions?

In our project (thumbv7em-none-eabihf target), we only needed to implement calloc and free. You should fix your compilation issue above first. Afterwards, the linker will fail and tell you which functions are missing. You can implement them in Rust then using FFI.

We have also already used mbedtls in a no_std configuration in std environments. There, we did not have to implement any low-level functions ourselves. I guess the Rust runtime just uses this low-level C interface itself so it is already available.

Answer 7 · 2023-03-15T10:26:01.000Z

Do I need to link mbedtls-sys to my project at all?

No, not manually. It is added automatically as a transitive dependency of the mbedtls dependency. If that one is configured with default-features = false, features = ["no_std_deps"], you are good to go (from the Rust side!)

= note: the x86_64-qkernel-13220628803078393756 target may not support the standard library
= note: std is required by once_cell because it does not declare #![no_std]
= help: consider building the standard library from source with cargo build -Zbuild-std

In my dependency graph (cargo tree), I do not see once_cell at all. Are you sure this error is caused by mbedtls? Maybe, you should try with a more minimal example first.

Will the functions strstr/strlen/strncpy/strncmp/strcmp/snprintf/ memmove/memcpy/memcmp/memset be written in rust or c?
Do you have any reference or example of how to implement these functions and where should I add those functions?

In our project (thumbv7em-none-eabihf target), we only needed to implement calloc and free. You should fix your compilation issue above first. Afterwards, the linker will fail and tell you which functions are missing. You can implement them in Rust then using FFI.

We have also already used mbedtls in a no_std configuration in std environments. There, we did not have to implement any low-level functions ourselves. I guess the Rust runtime just uses this low-level C interface itself so it is already available.

Many thanks. I resolved the compilation error. The crate hashbrown bring the once_cell into the project. For some reason, once_cell require std after I add mbedtls.

After removing the crate hashbrown , I can compile and run the program.

Answer 8 · 2023-03-15T10:31:39.000Z

Thanks @DrTobe. Closing this as completed, @yaoxin1995 feel free to reopen if assistance is needed.

Answer 9 · 2023-03-22T09:53:06.000Z

Thanks @zugzwang , I have to reopen it because I got compilation error when I tried to link my rust calloc and free to mbedtls.

@DrTobe I have a follow up question of how to link the rust version 'calloc' and 'free' in no_std environment.

I implemented my own version of 'calloc' and 'free' as following:

const ALIGNMENT: usize = 16;

#[no_mangle]
pub extern "C" fn calloc(number_of_elements: core::ffi::c_ulong, element_size: core::ffi::c_ulong) -> *mut core::ffi::c_void {


    // TODO: CHECK number_of_elements or element_size is null
    // 0 as *mut c_void;
    //core::mem::transmute(0) as *mut c_void
    if number_of_elements == 0 || element_size == 0 {
        return core::ptr::null_mut() as *mut core::ffi::c_void;
    }

    let total_size = number_of_elements * element_size;
    let layout = Layout::from_size_align(total_size as usize, ALIGNMENT).unwrap();

    let ptr;
    unsafe {
        ptr = alloc(layout);
    }

    let pointer_addr = ptr.addr();

    {
        let mut allocator_tracker = ALLCATOR_RECODER.write();
        allocator_tracker.memory_layout_records.insert(pointer_addr, layout.clone());
    }

    ptr as *mut core::ffi::c_void
}

#[no_mangle]
pub extern "C" fn free(ptr: *mut core::ffi::c_void) {
    if ptr.is_null() { return }

    let ptr = ptr as *mut u8;

    let ptr_addr = ptr.addr();

    let layout;
    {
        let allocator_tracker = ALLCATOR_RECODER.write();
        layout = match allocator_tracker.memory_layout_records.get(&ptr_addr) {
            Some(e) => e.clone(),
            None => return,
        };
    }

    unsafe{
        dealloc(ptr, layout);
    }
}

But for some reason, I got the compilation error "ld: failed to convert GOTPCREL relocation; relink with --no-relax".

Do you have any idea how to link my own version of calloc and free to mbedtls?

Answer 10 · 2023-03-22T10:00:51.000Z

Sorry, I do not know what your error is about. In my case, the calloc and free definitions start like this:

use core::ffi::{c_size_t, c_void};

#[no_mangle]
pub unsafe extern "C" fn calloc(nitems: c_size_t, size: c_size_t) -> *mut c_void {
[...]
}

#[no_mangle]
pub unsafe extern "C" fn free(ptr: *mut c_void) {
[...]
}

Maybe you can try the same function signatures.