use pwritev and preadv on linux
milahu opened this issue · 2 comments
https://docs.rs/tokio-file/latest/tokio_file/trait.AioFileExt.html#method.writev_at
https://docs.rs/tokio-file/latest/tokio_file/trait.AioFileExt.html#method.readv_at
https://github.com/vimpunk/cratetorrent/blob/master/DESIGN.md#vectored-io
https://old.reddit.com/r/rust/comments/kiah3q/i_wrote_cratetorrent_a_bittorrent_engine_in_rust/
Blocks are not buffered in memory but dumped straight to disk. Once all are written, the data is validated.
This has a nice property of not taking any memory. In fact, the only extra memory taken is the peer read buffer which is static per peer.
This was working well so far and is simple enough. However recently I added a storage trait where you can do whatever else - e.g. buffer (cache) blocks and write them with pwritev to reduce system calls.
That extra complexity and extra memory taken though should be justified - and I haven't seen yet why it would.
Won't do this, so closing. The current design of not buffering anything is perfectly fine, and may be even superior to the buffering approach which would allow for pwritev/preadv. Not that long ago pwrite and pread were added instead to allow lockless concurrent reads/writes to files, which should be good enough.