features missing to support iroh

[dignifiedquire]

What is the name of your project?
iroh

Please provide a link to your project (GitHub repository, crates.io page, etc).
https://github.com/n0-computer/iroh

What features would you like from zerocopy?

Deriving/implementing FromBytes and AsBytes for structs with trailing byte slices.

#[derive(Debug, Clone, PartialEq, Eq)] // Can't add FromBytes, AsBytes, FromZeroes here 
#[repr(C)]
pub struct RecordIdentifier<B: ByteSlice> {
    namespace: [u8; 32],
    author: [u8; 32],
    key: B,
}

This is to support reading these identifiers from redb.

[joshlf]

Thanks for the request!

We're currently implementing something that should support this in #29. You can try this out in our latest 0.8 alpha release. In particular, add #[derive(KnownLayout)] to the top of your struct, and then APIs with a KnownLayout bound such as FromBytes::ref_from should be available to you.

Please note that this is under active development - we're still adding support for some APIs, and APIs are subject to changes (both changes to type signatures and name changes) before we stabilize and release 0.8. That said, we'd love for you to kick the tires and let us know if it works for you or if there are things we should change/add before we release 0.8!

[dignifiedquire]

Thanks a lot, I am very close to having things work with the 0.8 branch, the only thing I am missing is a way to get IntoBytes working. I think in theory I could just use slice::from_raw_parts but obviously I would like to avoid having to add unsafe code to achieve this.

#[derive(Debug, PartialEq, Eq, KnownLayout, FromBytes, NoCell, Unaligned)]
#[repr(C)]
pub struct RecordIdentifier {
    namespace: [u8; 32],
    author: [u8; 32],
    key: [u8],
}

impl AsRef<[u8]> for RecordIdentifier {
    fn as_ref(&self) -> &[u8] {
        // How can I do this? 
        todo!()
        // self.as_bytes()
    }
}

[dignifiedquire]

I guess this feature is currently dependent on std::mem::size_of_val_raw :/

[dignifiedquire]

I wonder if a nightly flag could be added to zerocopy to allow for doing this when using a nightly compiler, as a stopgap until this gets stabilized.

[dignifiedquire]

For reference, here is a complete gist that I am using to test this out: https://gist.github.com/dignifiedquire/ea1219ba8de5c02904b2680844759864

[joshlf]

If you use #[repr(C, packed)], then you can derive IntoBytes directly. Here's an example from 0.7 (because that's what's available on the Rust Playground; note that AsBytes is only renamed to IntoBytes in 0.8):

use zerocopy::*;

#[derive(AsBytes)]
#[repr(C, packed)]
pub struct RecordIdentifier {
    namespace: [u8; 32],
    author: [u8; 32],
    key: [u8],
}

impl AsRef<[u8]> for RecordIdentifier {
    fn as_ref(&self) -> &[u8] {
        self.as_bytes()
    }
}

Basically what's happening here is that, without repr(packed), the IntoBytes derive isn't smart enough to know that your type will never have padding (padding in DSTs is weird - the trailing padding is calculated dynamically as a function of the trailing slice length). In this particular case, repr(packed) doesn't actually change anything because none of your field types have alignment greater than 1 anyway, but it's a guarantee that the IntoBytes derive knows how to understand.

[dignifiedquire]

Perfect, works as I need it. Thanks a lot for the great support. Feel free to close or keep upon until 0.8 is released.

[joshlf]

Of course, np! I'm gonna close since this is already on our roadmap, but feel free to open a new issue if other things come up.