Instancing Superpowered objects from custom WASM library

[petermlyon]

Hi,

I've been having success building a custom WASM library and linking in the superpowered.bc file in a manner similar to the example in statictest however I've hit an issue when trying to instance Superpowered objects/processing units.

The most minimal example of the issue can be seen by modifying main.cpp file in the statictest example as follows:

#include "Superpowered.h"
#include "SuperpoweredSimple.h"
#include "SuperpoweredWhoosh.h"
#include <emscripten.h>
#include <emscripten/bind.h>

static unsigned int testFunction() {
    Superpowered::Initialize("ExampleLicenseKey-WillExpire-OnNextUpdate", false, false, false, false, false, false, false);
    Superpowered::Whoosh *woosh = new Superpowered::Whoosh(44100);
    return Superpowered::Version();
}

EMSCRIPTEN_BINDINGS (TEST) {
    emscripten::function("testFunction", &testFunction);
}

This builds fine using build.sh but testFunction no longer returns a value, and instead I get Aborted() in the console when trying to instantiate a Whoosh.

I've tried with Emscripten 2.0.31/30, Sp SDK version 20400 and 20300, and have tested running ./build.sh on macOS and on Linux.

Peter

[gaborszanto]

The enable effects parameter of Superpowered::Initialize is false, therefore creating the Whoosh will abort.

[petermlyon]

🤦 Thanks Gabor!

[tralves]

@petermlyon did you figure out how to use your new WASM library in an AudioWorkletProcessor? Can you show how you did that?

Otherwise, @gaborszanto, it would be super-useful to have an example like the statictest but where the Superpowered input/output connects all the way to the AudioContext.

Thanks!

[petermlyon]

@tralves I did, but it was a while ago now and I don't have the original code left around. I think my first steps to get it running were something like:

add a process method to the main.cpp file (this is a loopback)

EMSCRIPTEN_KEEPALIVE void process(float* input, float *output, int buffersize) {
    std::copy(output, input, buffersize*2);
}

Change the output of the statictest build.sh to testmodule.wasm and build it

Then drop that into the example_guitardistortion/ folder, change Superpowered = await SuperpoweredGlue.fetch('./superpowered/superpowered.wasm'); (line 197 in main.js) to point to your new binary

then modify processor_live.js to

class MyProcessor extends SuperpoweredWebAudio.AudioWorkletProcessor {
    // runs after the constructor
    onReady() {
        console.log(this.Superpowered) // this is really useful for finding your methods which should be in this.Superpowered.wasmInstance.exports, I think the wrapper copies them to this.Superpowered.__functions__
    }

    onMessageFromMainScope(message) {
    }

    processAudio(inputBuffer, outputBuffer, buffersize, parameters) {
        this.Superpowered.__functions__.process(inputBuffer.pointer, outputBuffer.pointer, buffersize);
    }
}

if (typeof AudioWorkletProcessor === 'function') registerProcessor('MyProcessor', MyProcessor);
export default MyProcessor;

Then if you start up the guitar distortion example in 'live' mode, you should have hopefully achieved a loopback!

No guarantees that will actually work but it should be a good starting point :)

[tralves]

Thanks @petermlyon, you put me in the right direction.
Although, I am getting this error:

LinkError: WebAssembly.instantiate(): Import #0 module="env" function="emscripten_run_script" error: function import requires a callable

emscripten_run_script is a function that is created with the module on testmodule.js. Since I changed the output to .wasm, testmodule.js is no longer being created but for some reason it is still expected. SuperpoweredGlueModule.js doesn't provide this function, which looks like it could be related to my build.sh:

em++ --bind \
./main.cpp \
../superpowered.bc \
-I../../SuperpoweredSDK/Superpowered \
--no-entry \
-o testmodule.wasm

Any clues? I'm pulling my hair out...

[petermlyon]

No idea what the right answer to this is, but I think I copied the implementation over from testmodule.js to the exports list(s) in SuperpoweredGlue*.js

This is where my wrapper .js files started to diverge from the ones provided, maybe there is a more correct solution?

[gaborszanto]

Otherwise, @gaborszanto, it would be super-useful to have an example like the statictest but where the Superpowered input/output connects all the way to the AudioContext.

It doesn't "connect" at all. The included example projects set up AudioContext with some helper functions in SuperpoweredWebAudio.js, but you can also use vanilla standard Web Audio to create the AudioContext. The result is the same, standard Web Audio AudioContext and AudioNodes, etc.

The process() method of the standard AudioWorkletProcessor has buffers (typed arrays). You need to fill them with audio. Superpowered components read or write the contents of Linear Memory (standard WebAssembly feature), and the contents of Linear Memory can be copied into those buffers. There is no "magical connection", just plain simple data handling.

[tralves]

@gaborszanto I understand that. I meant that it would be useful to have a statictest example used in the context of an AudioWorkletProcessor, and even better with all the SuperpoweredGlue setup and helpers.

I don't know how you did it, @petermlyon, but I am getting into problems. The WASM module and instance loads and even the __demangle__ works, but the functions this.wasmInstance.exports._initialize(); and Superpowered.Initialize() aren't there. I am giving up and creating the worklet and processors without SuperpoweredGlue and SuperpoweredWebAudio.

[petermlyon]

@tralves That sounds like a reasonable approach!

For anyone else who bumps into this, I just did a thorough comparison of my build.sh with the one provided in statictest and I had to remove the --bind option to get the instantiate() method to show up on the wasm instance.

[gaborszanto]

statictest was created to show how to create your own custom WebAssembly binary.

It involves your own custom ways for data I/O and interfacing with your own custom functions. It needs a deeper understanding on how WebAssembly works (Linear Memory, functions), since you will provide your own custom "glue" (if any).

bind is Emscripten's way on doing this, but it's only one way from many.

[tralves]

Hi again!
So I built a working example project that uses a WASM binary linked with SP inside an AudioWorklet.
It includes playing a Sample (with AdvancedAudioPlayer), Reverb and a Generator.

Problem is that when I use SP, the output has noises and artefacts. Doesn't seem to be related to decoding or sample rates.

Can you please take a look (and a listen 🙂) @gaborszanto @petermlyon ?

Demo: https://elegant-keller-bbe4de.netlify.app/

Repo: https://github.com/tralves/sp-staticwasm-worklet

[petermlyon]

Hey @tralves,

You have some weird logic in your process call with effectively output[0].forEach(channel =>, you only need to call process once as the audio channels are interleaved in a single buffer by bufferToWasm

The following js process call works for me in your example (I fixed the frames-per-process-call size to 128 for simplicity as that is the size used by AudioWorkletNode)

   process(inputs, outputs, parameters) {
    if (!this.wasmInitialized || !this.samplesLoaded) return true;
    // not sure what you're channel.forEach was about here, output is interleaved into a single buffer.
    if (inputs[0].length > 1) this.bufferToWASM(this.inputBuffer, inputs);
    // this.processAudio(this.inputBuffer, this.outputBuffer, this.inputBuffer.array.length / 2, parameters);
    switch (this.mode) {
      case 'feedback':
        this.processFeedback(this.inputBuffer.pointer, this.outputBuffer.pointer, 256); // again, 128 * 2 channels
      break;
      case 'reverb':
        this.processReverb(this.inputBuffer.pointer, this.outputBuffer.pointer, 256);
      break;
      case 'sample':
        this.processSample(this.inputBuffer.pointer, this.outputBuffer.pointer, 256);
      break;
      case 'pink-noise':
        this.processWhiteNoise(this.inputBuffer.pointer, this.outputBuffer.pointer, 256);
      break;
      case 'pink-noise-sp':
        this.processWhiteNoiseSP(this.inputBuffer.pointer, this.outputBuffer.pointer, 256);
      break;
      case 'sound':
        this.processSound(this.inputBuffer.pointer, this.outputBuffer.pointer, 256);
      break;
    
      default:
        break;
    }
      
      // for (let i = 0; i < this.inputBuffer.array.length; i++) {
      //   this.outputBuffer.array[i] = this.inputBuffer.array[i];

      // }
      if (outputs[0].length > 1) this.bufferToJS(this.outputBuffer, outputs);
      // for (let i = 0; i < channel.length; i++) {
      //   channel[i] = this.module?.randomNoise() ?? 0;
      // }
    return true;
  }

[tralves]

🤦‍♂️🤦‍♂️🤦‍♂️🤦‍♂️🤦‍♂️🤦‍♂️🤦‍♂️
That's why you should have an extra pair of eyes...
That code came from the very early steps of creating the WorkletProcessor, which I grabbed from here: https://developer.mozilla.org/en-US/docs/Web/API/AudioWorkletNode#examples

I want to buy you a beer. Tell me how here or drop me an email (address in GH profile). For real.

[tralves]

@gaborszanto Hi!
I noticed the new version (2.5) does not include superpowered.bc. Will you be adding it (or some alternative) so we can create a WASM static lib?
Thanks