petersalomonsen/javascriptmusic

Fantastic project ... and a few questions

Opened this issue · 2 comments

brian-hay commented

Hi Peter and thanks for a fantastic project, posts and presentations.

I'm a JS/TS/3D dev but currently learning AssemblyScript, in part to do some retro C64, VIC-II & SID emulation with WASM. I'm a graphics guy and have rarely touched on audio, so am seeking advice and simple AS audio examples, the latter being hard to come by.

  1. I'd like to emulate SID digi playback but I understand that ideally requires cycle accurate emulation, which may not be possible if calling my WASM code or accessing shared memory on requestAnimationFrame. If I iterate faster than that in JS, even the high-resolution timers aren't guaranteed to be that accurate, plus I risk locking up the browser. What's a good approach here?
  2. Is it possible to generate audio directly in AS/WASM or can I only manipulate buffers on the WASM side then output audio on the JS side with the Web Audio API?
  3. Is there a simple AS audio example that you know of that generates a simple sine or square waveform at a high sample rate?
petersalomonsen commented

Hi, thanks for the feedback :)

Regarding your questions:

  1. The best approach here is to use AudioWorklet which is what I use in this project. AudioWorklet renders buffers of 128 frames and send to the audio hardware ( via a AudioWorkletNode ). This is the approach used in this project to obtain low latency and realtime playback.

  2. The AudioWorkletProcessor has to be written in JS but it can call into WASM for rendering the buffer of 128 frames.

  3. The heart of the code generating the audio in this project is the fillSampleBuffer method ( https://github.com/petersalomonsen/javascriptmusic/blob/master/wasmaudioworklet/synth1/assembly/index.ts#L173 ). This code of course takes into account that you should be able to control the notes from the outside and also use multiple instruments. In order to just make the simple basic example, you'd just make sure that your process method of the AudioWorklet calls into WebAssembly ( or even just does it in JS ) to something that just returns samples of a sine. Also read about AudioWorklet here: https://developer.chrome.com/blog/audio-worklet/

  • ❤️1
brian-hay commented

Thanks for the great advice and links @petersalomonsen !