Question regarding `tf.ready()` and `ml5.setBackend("webgl");` in NeuralNetwork examples

[jchwenger]

Hi there,

I came across the error on Chrome for this example:

engine.js:103 Uncaught (in promise) Error: Backend 'undefined' has not yet been initialized. Make sure to await tf.ready() or await tf.setBackend() before calling other methods
    at get backend (engine.js:103:19)
    at b.makeTensor (engine.js:612:35)
    at s (tensor_ops_util.js:66:19)
    at Module.a (tensor.js:199:12)
    at SD.formatInputsForPredictionAll (index.js:443:12)
    at SD.predictSyncInternal (index.js:883:28)
    at SD.predictSync (index.js:828:17)
    at finishedTraining (sketch.js:130:18)
    at callcallback.js:35:9

Which goes away if you add the following line, as you did in your current example:

  // For this example to work across all browsers
  // "webgl" or "cpu" needs to be set as the backend
  ml5.setBackend("webgl");

Curious about it, I did a quick search in the source code, and now I wonder if it might not be a case of just adding tf.ready() at the right spot in NeuralNetwork/index.js? I'm sorry I'm not yet fully set up to make this experiment. It seems that it's called in all the other classes but not this one:

[ziyuan-linn]

Hi @jchwenger ,

Thank you for submitting this issue!

TensorFlow.js automatically selects the webgpu backend on Chrome browser when the backend is not manually set. await tf.ready() is there for compatibility with the webgpu backend, which loads asynchronously.

Adding await tf.ready() is sufficient for other models, but adding it to NeuralNetwork will just trigger another error. This is because' NeuralNetwork' uses a few operations, such as arraySync, that are not supported in the webgpu backend. There is a draft PR #138 that adds webgpu support to NeualNetwork, but it would cause some breaking changes on the user's end.

Perhaps this is something we could look into toward the 2.0 release of p5.js, which adopts the promise/async/await interface.

We currently have the users manually select either webgl or cpu backend when they are using NerualNetwok. Perhaps one thing we could implement now is an automatic switch to the webgl backend when a NeualNetwork instance is created and the backend is webgpu, so the users don't have to manually set the backend.

[jchwenger]

Thanks so much @ziyuan-linn for the detailed response, once again it makes sense!

Read your exchange about the PR, I see the reasoning behind delaying having the async mechnism there for now.

Good to know in any case, looking forward to seeing how this will all develop!