Use const generics to implement Io trait for [T; N]

[Frago9876543210]

Also can be used for slice and together with #11

[ahmed-masud]

Is there a simple way to use endiannezz for [T; N]s without using const generics?

[Frago9876543210]

@ahmed-masud const generics already in stable rust anyway

[ahmed-masud]

Okay so something that produces something like this should do the trick correct? I am going to try and implement this :P wish me luck hehe

use num_traits as traits; // 0.2.14
use traits::PrimInt;
use traits::Num;
use std::convert::TryInto;

trait Primitive {
    type Buf;
    fn to_be_bytes(&self) -> <Self as Primitive>::Buf;
}


impl Primitive for u64 {
    type Buf = u64;
    fn to_be_bytes(&self) -> <Self::Buf as Primitive>::Buf {
        let primitive = *self as <Self as Primitive>::Buf;
        primitive.to_be()
    }
}

impl<T:PrimInt, const N: usize> Primitive for [T; N]
where
    T: Primitive + Sized + Num + Copy
{
    type Buf = [T; N];
    fn to_be_bytes(&self) -> <Self::Buf as Primitive>::Buf {
        let primitive = *self as <Self as Primitive>::Buf;
        let _r: Vec<T> = primitive.iter().map(|x| {
            x.to_be()
        }).collect();
        _r.try_into()
            .unwrap_or_else(|v: Vec<T>|
               panic!("Expected a Vec of length {} but it was {}", N, v.len()))
    }
}

fn main()  {
    let a: [u64; 5] = [0xdead; 5];
    println!("{:?}", a.to_be_bytes());
}

[Frago9876543210]

@ahmed-masud collect allocates memory in heap anyway. Also don't forget about [T: Primitive; N] and [T: Io; N] cases. I guess it should be handled as well

[Frago9876543210]

I'm going to reimplement this library without depending on std:: in future

[ahmed-masud]

collect allocates memory in heap anyway.

That is a good point—Rust was complaining about not being able to figure out the type of _r—but I'll watch for double allocs.

Also don't forget about [T: Primitive; N] and [T: Io; N] cases. I guess it should be handled as well

Thank you, that makes sense.

I am going to implement the related bits by hand first (without augmenting the derive for a simple case, I am also rather green with macros) and then see where it goes.

[Frago9876543210]

Why it should be implemented without const generics? They are stable since rust 1.51. The only excuse I can make against the implementation it with const generics is generating a lot of same code for different types.

[Frago9876543210]

Also I'd like to avoid panics and creating errors with messages using io::Error::new(_). See rust-lang/rust#82812 for details.
TL;DR: io::Error::new(_) allocates memory for errors 3-d times

[ahmed-masud]

Why it should be implemented without const generics? They are stable since rust 1.51. The only excuse I can make against the implementation it with const generics is generating a lot of same code for different types.

Oh I AM implementing it with const generics (but before i do the derive macro code generation, i wanted to do basically write out the code by hand).

Also I'd like to avoid panics and creating errors with messages using io::Error::new(_). See rust-lang/rust#82812 for details.

Yeah I wasn't planning on sticking any panics in the library, I should've clarified my code above is just "toy code" to ask you if I was on the right path :)

[Frago9876543210]

Would be nice to don't include any new deps such as num traits

[ahmed-masud]

There is however a small issue that I am currently running into, may be you can give me your thoughts on this; So I am starting by this implementation in the lib.rs right under impl_primitives![...] on line 221:

impl<T, const N: usize> Primitive for [T; N] 
where T: Sized + Copy + Primitive
{
    type Buf = [u8; mem::size_of::<T>() * N];
                            ^^^^^^^^^^^^^^^^^^ 
                            // I take it that T here should be  limited to the 
                            // numerical primitives and I should just stick this 
                            // template up in the macro and delegate this out?                                         
    fn to_ne_bytes(self: Self) -> Self::Buf {
        todo!()
    }

    fn to_le_bytes(self) -> Self::Buf {
        todo!()
    }

    fn to_be_bytes(self) -> Self::Buf {
        todo!()
    }

    fn from_ne_bytes(bytes: Self::Buf) -> Self {
        todo!()
    }

    fn from_le_bytes(bytes: Self::Buf) -> Self {
        todo!()
    }

    fn from_be_bytes(bytes: Self::Buf) -> Self {
        todo!()
    }

}

[ahmed-masud]

I think the way to do it without Num (because that was what I was using to limit the T to numerical primitives, is to use a copy of the macro that you use and simply implement the const generic array types. Let me implement something (may be a bit crude to start with but we can tighten it up) ...

[Frago9876543210]

You can implement Io for [T; N] instead of Primitive. E.g. bool doesn't have to_ne_bytes method and it's implemented as Io

[ahmed-masud]

You can implement Io for [T; N] instead of Primitive. E.g. bool doesn't have to_ne_bytes method and it's implemented as Io

How would that deal with something like [u32; 5]? I would expect that to have to_le_bytes() and to_be_bytes() ... that is to say:

[u32; 5].to_le_bytes: [u32.to_le_bytes(); 5]

[ahmed-masud]

Hey boss,

Check out the develop branch on my fork, https://github.com/safai-labs/endiannezz/tree/develop

I have implemented const generics for all primitive types (it's not possible right now to do it for non-primitive types because of the missing features in Rust).

There were some choices about inserting some panics and asserts in the code, which I documented in the comments. In real life, as the code stands, that branch should NEVER trigger because sized things should always match up. I could've simply unwrapped however, I like injecting a 'panic!' so that it can be grepped in case of internal bug.

Although the code may need some prettifying I think it is solid, and works rather nicely, and I am comfortable enough using it in production for a rather complex decentralized filesystem.

I also added the ability to "skip" attributes inside structures to allow for skipping of a field in a struct. It's skipped for write and filled in with Default:;default() for read.

It was necessary to create internal traits for AsRef, AsMut and Default which are not visible to or interacted with by crate users. This is because these need to be implemented for [T; N] which is always "external" to current crate.

I've added my name to the LICENSE file to enable redistribution. You can do a pull request as you see fit.

Cheers,

A

[Frago9876543210]

I did new branch that much simpler implements [T: Primitive; N] support: https://github.com/Frago9876543210/endiannezz/tree/const-generics
[T: Io; N] not supported yet. I think it was possible to do something like

#[derive(Io, Debug, PartialEq)]
#[endian(big)]
struct ParseMe {
    f: Seq<CountReader, Type>,
}

but first of all need to solve the problem with specialization. <Io xor Primitive>::{read, write} mega useful