Possibility of exposing a Trait as a vtable in current release?

Question

Possibility of exposing a Trait as a vtable in current release?

Closed this issue a year ago · 12 comments

I would like to take a Trait that I have and allow someone from the C side to provide an implementation of it in order to create a generic Rust struct internally. Based on the current documentation and my browsing of issues here I'm not sure I would accomplish that (even in a workaround type of way) and how that will improve with any changes that are coming.

Usage would be something like this

pub trait SomeTrait {
    fn some_method(&self, input: &[u8]) -> Result<Vec<u8>,  SomeError>;
    fn some_other_method(&mut self) -> String;
}

pub struct SomeStruct<T: SomeTrait> {
    inner_trait: SomeTrait
}

My assumption was that I would be able to do something like the following but I'm struggling with the specifics

#[ffi_export]
#[derive_ReprC]
pub struct SomeTraitVTable {
    some_method: extern "C" fn (self??, input: slice_ref<u8>,  out: Out???) -> c_int,
    some_other_method: extern "C" fn (&mut self??) -> repr_c::String
}

impl SomeTrait for SomeTraitVTable {
    // Not sure how this would work and how I would deal with &self and the output variable
}

#[ffi_export]
pub fn some_struct_new(repr_c::Box<SomeTraitVTable>) -> repr_c::Box<SomeStruct> {
    .....
}

Answer 1 · 2023-01-25T19:11:48.000Z

@danielhenrymantilla any thoughts?

Answer 2 · 2023-01-25T20:17:13.000Z

@tomleavy yes, this is definitely on safer-ffi's radar, and actually a feature that will be released rather soon:

#133

If you are feeling adventurous, feel free to patch safer-ffi with that branch and see how it fares.

The idea would be:

[dependencies.safer-ffi] # or [patch.crates-io.safer-ffi]
git = "https://github.com/getditto/safer_ffi"
# branch = "dyn-traits-part-2"
rev = "17e003eddd4088289d3bd93ab3d28c4c79090be6"  # HEAD of `dyn-traits-part-2` as we speak

#[derive_ReprC(dyn)]
pub trait SomeTrait {
    fn some_method(&self, input: c_slice::Ref<'_, u8>) -> repr_c::Vec<u8>; // repr_c::Result does not exist yet
    fn some_other_method(&mut self) -> repr_c::String;
}

let's paper over Result for the moment, although you could probably make do by hand-rolling a struct { status_code, payload }

You'll need the trait signatures to be made FFI-compatible by hand (matching safer-ffi's low-level current design, but the idea mid-term would be to have a kind of IntoFfi trait that would let us express that Vecs can be replaced by repr_c::Vecs, &[u8] with c_slice::Ref<'_, u8>, and so on); so you'll probably rather have to write this trait in an FFI-dedicated fashion:

#[derive_ReprC(dyn)]
pub trait FfiSomeTrait {
    fn ffi_some_method(&self, input: c_slice::Ref<'_, u8>) -> repr_c::Vec<u8>; // repr_c::Result does not exist yet
    fn ffi_some_other_method(&mut self) -> repr_c::String;
}

impl<T : SomeTrait> FfiSomeTrait for T {
    fn ffi_some_method(&self, input: c_slice::Ref<'_, u8>) -> repr_c::Vec<u8> {
        self.some_method(input.into()).into()
    }
    fn ffi_some_other_method(&mut self) -> repr_c::String {
        self.some_other_method().into()
    }
}

`VirtualPtr`

From there, the key wrapper type provided by safer-ffi which makes all the magic just click is VirtualPtr (in the prelude): a VirtualPtr<dyn 'lt + FfiSomeTrait> (feel free to add Send + Sync either as super-traits or inside that dyn trait) is then a ReprC type whose layout is that of:

#[repr(C)]
struct VirtualPtr<dyn 'lt /* + Send + Sync + */ + FfiSomeTrait> {
    ptr: *mut Erased,
    ffi_some_method: unsafe extern "C" fn(*const Erased, c_slice::Raw<u8>) -> repr_c::Vec_Layout<u8>,
    ffi_some_other_method: unsafe extern "C" fn(*mutErased) -> repr_c::String_Layout,
    ...
}

(another extension in the future would be to be able to customize the indirection of the vtable; for now an inlined vtable is just simpler across FFI).

And some convenience impls:

VirtualPtr<dyn ... + FfiSomeTrait> : ... + FfiSomeTrait ¹
VirtualPtr<dyn ... + FfiSomeTrait> : From<Box<T>> where T : ... + FfiSomeTrait
- (there are even From<&'lt mut T> impls for VirtualPtr<dyn 'lt + ...>)

You can see a VirtualPtr<dyn ...> as a Box<dyn ...>, except for the fact it even virtualizes the fact of being a Box: any pointer matching the requires API suits!

for instance, there is actually a
```
impl<'lt, T : ... + FfiSomeTrait> From<&'lt mut T> for VirtualPtr<dyn 'lt + ... + FfiSomeTrait>
```
since when within a 'lt lifetime, &'lt mut dyn is just as capable as Box<dyn 'lt +

For more info, it turns out the language is currently considering adding, as a fully stand-alone abstraction, something that matches VirtualPtr conceptual design (but for the ffi considerations) quite closely: dyn *.

So, back to our example, we'd probably also want to write:

impl<'lt> SomeTrait for VirtualPtr<dyn 'lt + FfiSomeTrait> {
    fn some_method(&self, input: &[u8]) -> Vec<u8> {
        self.ffi_some_method(input.into()).into()
    }
    fn some_other_method(&mut self) -> String {
        self.ffi_some_other_method().into()
    }
}

See also, how the feature itself is being used to model Futures and Executor Handles (Spawn abstraction, technically) by safer-ffi itself: https://github.com/getditto/safer_ffi/blob/17e003eddd4088289d3bd93ab3d28c4c79090be6/src/dyn_traits/futures/executor.rs

due to coherence reasons, this may be "duck-typed" rather than an official impl. ↩

Answer 3 · 2023-01-25T20:23:30.000Z

Back to a high-level view of the project, I should have this properly merged in master / released to crates.io within a month or two, hopefully with a quite lengthy book / guide about these things (my previous post may be a bit too densely packed with info, and lacking some examples)

Answer 4 · 2023-01-25T20:43:25.000Z

This is awesome, thanks @danielhenrymantilla . Question about VirtualPtr, is that the type I would actually use when I want to utilize the trait someplace? My example calling new on SomeStruct that requires T: SomeTrait would be like so?

#[ffi_export]
pub fn some_struct_new(input: VirtualPtr<dyn '_ + FfiSomeTrait>) -> Box<SomeStruct> {
    Box::new(SomeStruct::new(input))
}

Answer 5 · 2023-01-26T16:12:26.000Z

Yeah, that's the idea 🙂

Answer 6 · 2023-01-31T16:27:42.000Z

Using the https://github.com/getditto/safer_ffi/tree/dyn-traits-part-2 branch, I defined the following trait but I can't seem to get the corresponding vtable definition in the generated C header:

pub trait ExampleTrait {
    fn method_ref(&self, s: &str, i: i32) -> String;
    fn method_mut(&mut self, s: &str, i: i32) -> String;
}

#[derive_ReprC(dyn)]
pub trait FfiExampleTrait {
    fn method_ref(&self, s: char_p::Ref<'_>, i: i32) -> repr_c::String;
    fn method_mut(&mut self, s: char_p::Ref<'_>, i: i32) -> repr_c::String;
}

I couldn't find any use of inventory for trait definitions in the proc-macro. Is generating vtables in the C header not implemented yet or am I missing something?

Thank you.

Answer 7 · 2023-01-31T17:02:31.000Z

@stefunctional this may be because the FFI type that makes use of said vtable is VirtualPtr<dyn FfiExampleTrait + ...>; and safer-ffi is currently "lazy" w.r.t. what makes it to headers:

functions always make it to the header / are "ffi-exported";
then anything transitively referred to by something ffi-exported (e.g., FFI types in the signature of a function) are then themselves ffi-exported.

So try adding:

// any of the two types involved should trigger the vtable definitions
#[ffi_export]
fn lets_see(
    _a: VirtualPtr<dyn 'static + Send + Sync + FfiExampleTrait>,
    _b: VirtualPtr<dyn '_ + FfiExampleTrait>,
)
{}

Answer 8 · 2023-01-31T17:19:40.000Z

That was it. Thank you very much for the quick answer!

Answer 9 · 2023-02-06T12:38:42.000Z

Do you need help to get the dyn-traits-part-2 branch to a point where it can be merged? It seems to be working well in the little experiment we conducted and we would be happy to help move this forward.

Answer 10 · 2023-03-16T12:27:35.000Z

Hey @stefunctional sorry for the late reply 🙇 I appreciate the offer, truly, and may take you up on it for later ideas 😄 but in this case the remaining effort was mostly:

around documentation, which I've finally tackled,
and then around cleaning my draft code with master and with crates.io-releasability to get everything in order,

and both of these tasks were thus about dealing with my messy code, so I was the best suited for them.

I have been quite busy with other stuff this past month, hence this being a bit more delayed than anticipated, but now all this has been released as 0.1.0-rc1 to crates.io

Answer 11 · 2023-03-16T12:58:21.000Z

I'm gonna close this since #155, in a way, finished fixing it, but we can keep discussing about dyn traits here or over the Discussions 🙂

Answer 12 · 2023-03-16T16:59:20.000Z

No worries. Thank you very much for your work on this. I just skimmed through the change log and I am excited to try out this new version.

VirtualPtr

Footnotes

`VirtualPtr`