Intrinsic (natural) size of WebGL canvas is unclear

[kainino0x]

Specifically in the case where drawing buffer size and canvas size don't match (which already gets into unspecified behavior), the WebGL spec is unclear about the intrinsic (natural) size of the canvas. This is important for at least 2 things:

• The default CSS size
• The drawImage() destination size

  If not specified, the dw and dh arguments must default to the values of sw and sh, interpreted such that one CSS pixel in the image is treated as one unit in the output bitmap's coordinate space. If the sx, sy, sw, and sh arguments are omitted, then they must default to 0, 0, the image's intrinsic width in image pixels, and the image's intrinsic height in image pixels, respectively.
  [spec]

Chrome and Firefox agree about the first thing: the default CSS size is the canvas size, not the drawing buffer size.
However, for drawImage() destination size, Chrome uses the canvas size, while Firefox uses the drawing buffer size.
Test case: https://codepen.io/kainino0x/pen/dyzqXym

Since this is stepping into unspecified territory, it's understandable that they differ. However there is this non-normative note in the WebGL 1.0 spec which almost seems to answer the question:

The constraint above does not change the amount of space the canvas element consumes on the web page, even on a high-definition display. The canvas's intrinsic dimensions [CANVAS] equal the size of its coordinate space, with the numbers interpreted in CSS pixels, and CSS pixels are resolution-independent [CSS].

1. What is "the size of its coordinate space"?
2. Why is this text non-normative? The HTML canvas spec doesn't seem to answer it either, so it probably needs to be answered here - I think HTML only defines the intrinsic size for context modes none (and 2d presumably), and sort of for placeholder.

[kainino0x]

FWIW on the distinction between "intrinsic" and "natural" sizes:

Renamed replaced element “intrinsic” dimensions to “natural” dimensions in order to avoid confusion with intrinsic sizes (see Issue 4961). [spec]

Replaced elements frequently derive their intrinsic size from their natural dimensions. [spec]

[greggman]

What does chrome do to use the canvas size? Does it bilinear blit the smaller drawingbuffer contents stretched to the size of the src canvas to the dst canvas?

Also, this is what I see on mac

Firefox

Chrome

which suggests they're doing something even more different?

If I set the css to

canvas { border: 1px solid black; width: 100%; height: 4px }

Then I can more easily see Chrome's behavior

And for Firefox

Safari matches Chrome

[kainino0x]

What does chrome do to use the canvas size? Does it bilinear blit the smaller drawingbuffer contents stretched to the size of the src canvas to the dst canvas?

Most likely, as this is what would happen if you explicitly set the dw/dh parameters to a size larger than the canvas.

which suggests they're doing something even more different?

Firefox preserves the aspect ratio while Chrome doesn't. Both are allowed by the spec.
Then I'm pretty sure what's happening is:

• Chrome then takes the 16384x4 drawing buffer and draws it into a 24576x4 destination rectangle.
• Firefox takes the 6144x1 drawing buffer and draws it into a 6144x1 destination rectangle.

This becomes clearer if you fiddle with the resolution of the destination canvas. I think Firefox might also be downrezzing the 2d canvas drawing buffer so the red line disappears. E.g. if you make cvs1 8x8 then Firefox shows a 1px red line while Chrome shows a 4px red line.

[toji]

Firefox preserves the aspect ratio while Chrome doesn't. Both are allowed by the spec.

Chrome did preserve the aspect ratio at one point. That's how I wrote it when we did some of the initial code to scale down the default framebuffer when encountering OOMs or (in some cases) hard coded limits. I'm not sure when or why that would have changed, but it seems potentially problematic. If developers are using the drawingBufferWidth/Height for things like viewports (which they should) then it's not unreasonable for them to also use it for determining aspect ratios for projection matrices.

[toji]

Ah, seems like the murderer was me!

In https://chromiumcodereview.appspot.com/13951020/ (way back in ye' old 2013) I did a refactor that made the drawing buffer size clamp either dimension if it was larger than the max texture size for the system. This happened before any scaling, which does still preserve aspect ratio if you hit memory limits.

The clamp appears to replace some older logic where the texture simply wouldn't allocate if it exceeds the max texture size, so it was an improvement in that regard, but I didn't leave any comments regarding why I chose to clamp and ignore aspect ratio rather than scale both dimensions proportionally. Probably just didn't think about it at the time.

So there you go, Chrome's behavior is likely a historical accident rather than anything intentional. I wouldn't have any qualms if we wanted to normalize on Firefox's behavior, since it seems a bit more principled. It would technically be a breaking change, but any developers relying on this exact behavior are probably already broken cross-browser and just don't know it.

[kainino0x]

If we want to specify the behavior here, preserving the aspect ratio (as Firefox does) makes sense.

As for the natural dimensions and intrinsic size, I'm trying to work this out in WebGPU (gpuweb/gpuweb#2300). My current proposal, ported back to WebGL, would be:

• Intrinsic size always matches the canvas .width/.height
• Natural dimensions always match the drawing buffer dimensions

[greggman]

Does preserving the aspect ratio make sense?

If we copy <img> semantics, AFAIK the img.width and img.height have zero influence on the data I get out of image when drawing with it. The image has its naturalWidth, naturalHeight (the size in the file) and it has a user size (width, height). naturalWidth, naturalHeight seem equivalent to drawingBufferWidth and drawingBufferHeight

https://jsgist.org/?src=4355ee3c5f8f2fab68cbdb08789ebb29

Here I have a 256x256 pixel F texture. I've set its width and height as in <img src="f.png" width="32" height="128"> (right) but that has no affect on how it's drawn with drawImage (left). It still draws 256x256 which is what i'd expect. I feel like the same should be true of canvas. If the canvas is width=10, height=10000 but the drawingBufferWidth is 2048x2048 I'd expect drawImage to draw 2048x2048, the actual content, just like it did with img.

[kdashg]

Here I have a 256x256 pixel F texture. I've set its width and height as in <img src="f.png" width="32" height="128"> (right) but that has no affect on how it's drawn with drawImage (left). It still draws 256x256 which is what i'd expect. I feel like the same should be true of canvas. If the canvas is width=10, height=10000 but the drawingBufferWidth is 2048x2048 I'd expect drawImage to draw 2048x2048, the actual content, just like it did with img.

I think this is a good comparison. And it sounds like Firefox has this behavior already? I don't remember writing that logic, but maybe then it's no surprise that I think this is a good comparison. :)

One resolution then would be to change that (currently non-normative) note, and also test it and make it normative.

[kainino0x]

I hadn't thought about the fact that image elements can have different width/height attributes and internal size, and must have missed that in Gregg's comment. It's a good observation that I'll need to actually remember this time.

Looking into the HTML spec for a minute, image elements have .naturalWidth/.naturalHeight which "return the intrinsic [natural] dimensions of the image" (underlying data, not img element). Given drawImage is supposed to use the intrinsic [natural] dimensions, this matches up.

I think this makes <img> one of the exceptions to this text:

Replaced elements frequently derive their intrinsic size from their natural [intrinsic] dimensions. [spec]

because its intrinsic size is .width/.height (if set) while its natural dimensions are .naturalWidth/.naturalHeight.

Tangentially related, but interesting, fact from the MDN:

Note: Most of the time the natural width is the actual width of the image sent by the server. Nevertheless, browsers can modify an image before pushing it to the renderer. For example, Chrome degrades the resolution of images on low-end devices. In such cases, naturalWidth will consider the width of the image modified by such browser interventions as the natural width, and returns this value.

Which is actually very similar to automatic reduction of the drawing buffer size - and also seems rather hazardous for WebGL content (if it can run on the device at all).

---

The intrinsic [natural] dimensions of the canvas element when it represents embedded content* are equal to the dimensions of the element's bitmap. [spec]

(* true if it has a context, AFAIU)

So to try to distill the question into spec formalities, it's whether a WebGL canvas has:

1. intrinsic size = .width/.height, but
   natural dimensions = bitmap size = .drawingBufferWidth/.drawingBufferHeight.
   (Like <img>.)
2. intrinsic size = natural dimensions = .width/.height, but
   bitmap size = .drawingBufferWidth/.drawingBufferHeight.
   (This would conflict with the HTML spec quote above.)
3. intrinsic size = natural dimensions = bitmap size = .width/.height, and
   only the drawing buffer as seen by the WebGL context is .drawingBufferWidth/.drawingBufferHeight.
   (Internally, the "bitmap" is just represented by fewer pixels than its size would seem to indicate.)

Option 1 seems like the natural choice, but it also means implicit downsizing can substantially impact the behavior of drawImage.
Option 3 admittedly sounds ham-handed, but doesn't have that problem.

(There are parallels into WebGPU as well, which is what I'm more imminently concerned with: gpuweb/gpuweb#2416)

Also, in all cases, this nearby spec text would need to be clarified as not directly controlling the bitmap size:

The canvas element has two attributes to control the size of the element's bitmap: width and height.

[greggman]

Sorry, commenting on old comment but ...

If developers are using the drawingBufferWidth/Height for things like viewports (which they should) then it's not unreasonable for them to also use it for determining aspect ratios for projection matrices.

In general you should use canvas.clientWidth, canvas.clientHeight for projection matrices (or equivalent (element.getBoundingClientRect, ResizeObserver). The aspect you want for a projection matrix is the display aspect, which has nothing to do with how many pixels are in the the canvas (drawingBufferWidth, drawingBufferHeight)

https://jsgist.org/?src=022725acacec1ecc87b0960da8f259ea