rust-lang/rfcs

Proposed refactoring to implement core::io

fluffysquirrels opened this issue · 12 comments

fluffysquirrels commented

Label: T-libs

There have of course been several efforts to refactor std::io to make it usable without std as core::io or similar.

Some links to prior art from a few hours of searching:

From what I can see all efforts are currently stalled, but I'd like to make progress on using std::io in a no_std environment. My use case is a crate I'm writing that uses std::io::{Read, Write}, and I want it to build with std and when no_std.

As mentioned in https://internals.rust-lang.org/t/refactoring-std-for-ultimate-portability/4301, there are circular dependencies between std::io and the rest of std.

I have an idea that I've not seen mentioned: use conditional compilation in the std::io module to enable use without std. Steps required in more detail:

  1. Move rust/src/libstd/io/*.rs to a new directory rust/src/libio_template
  2. Add a new rust/src/libstd/io/mod.rs with just an include!("../../libio_template/mod.rs").
  3. Add new crate rust/src/libio_nostd, which will also include! libio_template.
  4. Add cfg feature "libio_for_std" to both libstd and libio_nostd, with it enabled and disabled by default respectively.
  5. Patch libio_template to compile in libstd and libio_nostd.
  6. All done. no_std consumers can use libio_nostd, libstd API should be identical, future std::io patches must be correct both in libstd and libio_nostd.

The ideal solution, as proposed in https://internals.rust-lang.org/t/refactoring-std-for-ultimate-portability/4301, is probably a careful refactoring to decouple std, but that seems like a lot of work, which has already stalled once. A templated approach allows quick progress and can be eradicated progressively as more parts of std::io are refactored out of std.

I'm willing to work on a prototype branch or more formal RFC if there is interest from the libs team.

TimNN commented

One problem I see with the include! approach is that std::io::Read and core::io::Read would be different traits, making APIs using either incompatible with each other.

  • 👍6
fluffysquirrels commented
Ericson2314 commented

I like the blanket impl I propose here https://internals.rust-lang.org/t/refactoring-std-for-ultimate-portability/4301/3 to connect the two traits.

  • 👍1
fluffysquirrels commented

@Ericson2314 Yes! I Read that thread with great interest.

I agree the platform abstraction layer sounds like a great approach. I didn't feel up to the task of starting that work and was looking for a quicker fix that could be improved later.

Edit: misread your post.

Yes, it should be possible to rewrite std::io as a concrete layer on top of a generic core::io layer with some wrapper structs to convert between the two, and make it all low-cost. I would be interested in prototyping that approach instead of what I proposed here.

  • 🎉1
SimonSapin commented

There’s also methods like Read::read_to_vec that mention Vec which is defined in the alloc crate.

Ericson2314 commented

@fluffysquirrels I think you closed by mistake?

@SimonSapin Indeed, I imagine we'd want an alloc::io, too.

fluffysquirrels commented

@fluffysquirrels I think you closed by mistake?

I was thinking the linked page may be more appropriate, but now I see your comments were on a forum, not an issue tracker. My mistake.

Regarding Read::read_to_vec and similar, perhaps core::io could have a feature flag to toggle features that require alloc.

  • 👍1
Ericson2314 commented

@fluffysquirrels Sure, fine to continue this elsewhere. Thanks again for kicking off the discussion :).

lygstate commented

What's the situation of this?

  • 👀1
mauro-balades commented

What's the situation of this?

ded