scottstensland/websockets-streaming-audio

stream from server to browser an endless set of new input wav audio files

scottstensland opened this issue · 2 comments

scottstensland commented

to handle an upstream process which is synthesizing wav audio on server side which need to get rendered as audio on browser ... lets look at nodejs stream pipes

guest271314 commented

Re https://stackoverflow.com/questions/60782071/streaming-audio-through-websockets-to-web-audio-player#comment107769145_60782071

now building a new project so yes I'm in the same boat ... ping me github.com/scottstensland/websockets-streaming-audio

at Streaming audio through WebSockets to Web Audio player

  1. Produces audio on a server in to a 1 second WAV file
  2. Reads the WAV file and sends it through a websocket

AudioWorkletNode can be substituted for AudioContext.decodeAudioData() and AudioBufferSourceNode at

  1. Websocket sends the binary data to AudioContext.decodeAudioData
  2. Decoded audio is buffered until 4 packets (4 seconds)
  3. Buffer is processed and sent to AudioBufferSourceNode.start(time) where time = (clip_count * duration)

to avoid memory consumption issues. The implementations might be different at Chromium or Chrome and Firefox or Nightly. Nightly does not currently implement ReadableStream.pipeTo(),

FWIW see

Two versions of the code so far, tested at Chromium 81 and 82 launched with --enable-experimental-web-platform-features flag

Break tests welcome https://gist.githubusercontent.com/guest271314/1fcb5d90799c48fdf34c68dd7f74a304/raw/c06235ae8ebb1ae20f0edea5e8bdc119f0897d9a/UseItBreakItFileBugsRequestFeaturesTestTestTestTestToThePointItBreaks.txt.

guest271314 commented

Note, the code at https://github.com/guest271314/AudioWorkletStream/blob/master/audioWorklet.js was conceived of and is written to handle an infinite input stream of WAV file(s) or binary data capable of being converted to Float32Array's without a WAV header https://github.com/guest271314/AudioWorkletStream/blob/master/audioWorklet.js#L55.

Would suggest to process the input data at the server, if possible, to stream Float32Arrays having length set to exactly 128, save for the last part of the file, unless the stream is infinite, in which case the input read by WebSocket at client should always be Float32Array with length set to 128.