Support byte-array transfer in JS Interop

[AutumnEvans418]

Describe the bug

I am using Quaggajs to read barcodes and want to the image data from the canvas, but when I do this the blazor client disconnects and the JsInterlop throws a canceled task exception. I assume it's because the data is too large?

To Reproduce

This is the code I am using

function GetPicture() {
    var canvas = Quagga.canvas.dom.image;
    var ctx = canvas.getContext("2d");
    return ctx.getImageData(0,0,canvas.width,canvas.height).data;
}

[JSInvokable]
public async void BarcodeFound()
{
    await JSRuntime.InvokeAsync<byte[]>("GetPicture");
}

Further technical details

• ASP.NET Core version 3.1
• Include the output of dotnet --info

.NET Core SDK (reflecting any global.json):
 Version:   3.1.101
 Commit:    b377529961

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.18362
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\3.1.101\

Host (useful for support):
  Version: 3.1.3
  Commit:  4a9f85e9f8

.NET Core SDKs installed:
  2.2.207 [C:\Program Files\dotnet\sdk]
  2.2.402 [C:\Program Files\dotnet\sdk]
  3.1.100 [C:\Program Files\dotnet\sdk]
  3.1.101 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.14 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.14 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.14 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.0.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

• The IDE (VS / VS Code/ VS4Mac) you're running on, and it's version
  Visual studio 2019 professional 16.4.2

[AutumnEvans418]

The error I get in the console is this:

Error: Connection disconnected with error 'Error: Server returned an error on close: Connection closed with an error.'

Error: Cannot send data if the connection is not in the 'Connected' State.

[mkArtakMSFT]

Thanks for contacting us. byte arrays are not yet supported when doing interop. We will consider adding support for it in the future.

[SteveSandersonMS]

And to clarify why, it's because JS interop uses JSON as the wire format, and JSON has no standard way to represent byte arrays. You can send/receive base64 encoded byte arrays today. We'll look into automatic handling of raw byte data in the future.

[AutumnEvans418]

@SteveSandersonMS when I use toDataUrl() on the canvas I still get the same error, which is a base64 encoding.

[SteveSandersonMS]

In that case I think your issue is unrelated, because there's nothing special or unusual about passing strings via JS interop - that has always worked.

I suspect some other code on either the .NET side or the JS side is throwing, so I'd recommend looking at your server-side console output.

[msftbot]

We've moved this issue to the Backlog milestone. This means that it is not going to be worked on for the coming release. We will reassess the backlog following the current release and consider this item at that time. To learn more about our issue management process and to have better expectation regarding different types of issues you can read our Triage Process.

[msftbot]

Thanks for contacting us.
We're moving this issue to the Next sprint planning milestone for future evaluation / consideration. We will evaluate the request when we are planning the work for the next milestone. To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.