CanvasRenderingContext2D: Add getImageDataAsync method
Opened this issue · 6 comments
What is the issue with the HTML Standard?
The existing getImageData method is synchronous.
In practice, canvases are draw asynchronously (often by a GPU, but contention for the Javascript main thread can be avoided by doing CPU drawing on a non-main thread). This means that getImageData must stall the main thread while it reads back data. This has been slightly mitigated by willReadFrequently, but that's not without its own issues.
A better solution would be to create an asynchronous getImageData function. Something to the effect of
Promise<?> getImageDataAsync(
ImageData imagedata,
[EnforceRange] long sx,
[EnforceRange] long sy);
This will get a snapshot of the canvas at the moment that the getImageDataAsync call is made (it will not be affected by subsequent draw calls). The ImageData to populate is passed as a parameter, as opposed to being allocated by the call. This allows less memory thrashing, and it also removes ambiguity about "what should the default readback pixel format or color space be", because the ImageData already answers that.
Brillant move. I like the async method, even if it presents a very slight increase in difficulty in workflows.
Performance AND battery efficiency is a simultaneous win-win with this async verison of API, especially with HDR pixel formats (48-bit / 16bpp via Float16)
Glad to see this coming back.
One aspect I'm unsure about though is the timing of the API.
It's unclear when an author can expect the image will be available. I guess that most of the time it would be after the next paint, but I'm not sure that'd always be the case. Would it be ensured somehow that it would be ready before the next frame's painting time?
Should two simultaneous calls to getImageDataAsync() always resolve in a deterministic order, whatever the options?
Should any of it even be specced at all? I suppose for most use cases it's ok to be at one frame late, but having variable timings might cause issues for a bunch and might prevent the adoption of this feature.
I'd naturally expect that such an API always does that. Generally, this is kind of "Get this image once it's unblocked", and unblocking always happens at least once per paint cycle.
It's my understanding that getImageData blocks until the framebuffer is delivered -- so naturally I'd expect a nonblocking getImageDataAsync would report a callback right at the moment of getImageData unblock.
However:
That being said, it would be a good idea to explicitly mention it in the specifications, to assauge fears of a multi-frame latency added.
Minor nit: I kinda like the getImageDataInto() name I came up with in #5707. For parity with encodeInto() from the Encoding API.
Though encodeInto is sync, wouldn't this create confusion somehow?
That would confusingly mean getImageDataAsyncInto() or getImageDataIntoAsync() but that becomes longwinded. The "get" prefix and "into" suffix is usually designed in APIs independently, whether you're getEncoding() or encodeInto(), rather than getEncodingInto()
Either way, I'm neutral in this case. I think @ccameron-chromium is running with the getImageDataAsync() ball.