whatwg/html

ImageData: Add Float16Array backing support

Opened this issue · 2 comments

ccameron-chromium commented

What is the issue with the HTML Standard?

The data backing an ImageData can only be Uint8 RGBA.

This issue is to add the ability to create an ImageData with different formats, including Float16Array RGBA.

BlobTheKat commented

For completeness

Channel types:

RGB
RGBA
LUM (one value for all of RGB)
LUMA (one value for RGB, one for A)

Scalar types:

Uint8 (normalized)
Uint16 (normalized)
Float16
Float32

Given that APIs using ImageData do not support custom shaders I see no call for supporting

  • Integer formats
  • Higher precision formats
  • Component swizzling
  • RED/RG data
mdrejhon commented

Great step forward!

In the use case of professionals who are using web-based scientific/measurement tests of HDR displays (e.g. TestUFO), I eagerly anticipate this support. I have one suggestion though, to cover in the test cases.

Suggested Test Case: Make sure HDR Float16 pixels return the same numbers in WebGPU/WebGL and Canvas2D*

during bitmap-to-texture-to-bitmap conversion; at least when all reprocessing is turned off (via any flags we have access to), e.g. zero filters at browser level and system level

Also you want to create use cases to compare the behaviors / conversion between Canvas2D to WebGL/WebGPU (e.g. textures vs bitmaps), to make sure that it doesn't distort (when processing/tonemapping is at default and same in all, and you don't install any driver based color reprocessing utilities). Whereupon, converting a texture (WebGL/WebGPU) to a bitmap (Canvas2D), and vice versa -- pixel values remain same.

Some future TestUFO screens beyond its existing ~40 is going to render the same simple display-testing graphics to Canvas2D, WebGPU and WebGL via abstraction layers so feature parity (such as getting Float16 pixel format in all).

Seeing different WCG/HDR results in all three types of canvases will be weird, so it's good to benchmark for WCG/HDR equality in all types of canvases for full convertability (e.g. loading and converting each other's framebuffers, bitmaps or textures, and still seeing it result the same when configured properly for least reprocessing).

So someone might wish to have test cases that compares WebGPU vs Canvas2D (at least for whatever HDR extensions are added to which) to prevent accidental distortion of the pixel values (e.g. color shifts and whatnot). WebGL can be omitted, but I include it for legacys' sake and there's already HDR path there, so might as well include it.