Zero copy support via BYOB (add an example)

[aboba]

QUIC implementations such as quant support zero-copy (via warpcore). Via BYOB, zero-copy should be feasible for both readable and writable streams.

So it seems to me that zero-copy should be feasible in WebTransport via BYOB, in which case some examples should be provided to show how it works.

[DavidSchinazi]

Can you clarify what data exactly should not be copied and when? Our QUIC implementation lives in a different process from where JavaScript runs, so a copy will be necessary for security reasons.

[yutakahirano]

With Chromium implementation BYOB doesn't reduce copy but it does reduce the number of allocations, pressure for the GC system and the peak memory usage.

The allocation CPU cost is real, from our experience in WebSocket.

[aboba]

Going over the current state of browser implementations, a WebTransport application using WebAssembly and Workers may require as many as a dozen memory copies on both the sending and receiving side. There are copies for send and receive (C++/JS), for use of WebAssembly, as well as for use of data within a Transferrable Stream.

[yutakahirano]

Could you elaborate on a bit more please? We, the Streams API team in Google, want to help people write efficient applications by providing good APIs and implementation. cc:@ricea @domenic

[aboba]

One of the scenarios for WebTransport + WebCodecs is realtime communications in which portions of the pipeline (such as packetization or machine learning) are handled in WebAssembly. Currently, it appears that copies may exist at every stage in the pipeline (e.g. capture, machine learning, encode, packetization, send, receive, de-packetization, decode, more machine learning, render). Several of these copies represent additional overhead not present in WebRTC (e.g. no additional copies for packetization/de-packetization). This isn't solely a WebTransport problem; some of these copies are being worked on (support for zero-copy capture and zero-copy rasterization), some are due to WASM.

[aboba]

Is BYOB supported currently? If I try below code in Chromium, last line produces an error.

const reader = transport.incomingUnidirectionalStreams.getReader();
const {value, done } = await reader.read();
const byob_reader = value.getReader({mode: "byob"});

If it is supported, is there a running sample code?

[jianjunz]

BYOB support for WebTransport streams was merged just 10 days ago.

https://chromium-review.googlesource.com/c/chromium/src/+/3799089

[aboba]

Thanks! Will check it out in Canary.

[aboba]

@jianjunz Trying it out in M108, seems to be working very well. Overall latency is down quite a bit. Thank you!

[jianjunz]

Thanks for your feedback. BYOB support for datagrams (CL) is not merged yet. One concern from Adam is it will lose the datagram boundaries. Feel free to add your comments if you have. Thanks.

[jan-ivar]

The streams spec has a byob example we might want to crib.

[aboba]

I have included BYOB for streams in the WebCodecs Echo sample (requires M108+).

[jan-ivar]

Once #487 is merged, we have BYOB for both datagrams and regular streams. So now's a good time to add some examples to show how it works.