Using with AudioWorklet?
Opened this issue · 6 comments
Hey, I'm curious to try and use paulstretch.js but since ScriptProcessorNode is deprecated what would be the easiest way to use paulstretch.js with AudioWorklet?
In your opinion, could we somehow use it on AudioBuffer simply calling it with AudioWorklet class or the whole project must be rewritten to extend AudioWorkletProcessor and the calculations should be in process() function? Maybe you have some examples? Thank you! :)
Hi @Catsvilles ! The lib is actually implemented as a simple class which has methods to do the processing :
paulStretch = new PaulStretch(numberOfChannels, ratio, winSize)
paulStretch.write(data)
paulStretch.process()
paulStretch.read(arrayOut)
Therefore, there is nothing stopping you right now to put it in an AudioWorklet. Please let me know if you do it! I'd be happy to see it in action, and why not adding it as an example to the repo :)
@sebpiq Thanks for getting back to me! Absolutely, if I will hack this together one of those days I will submit pr or something :)
@sebpiq Hello again!
I've spent many hours trying to make Paulstretch.js work with AudioWorklet but unfortunately after solving one problem after another it seems that the core of Paulstretch.js is the possible problem for now as P.js requires the bufferSize to be no less than 4096 while AudioWorkletProcessor can output only 128 frames.
(I tested with lesser bufferSize with ScriptProcessorNode and it has too audible artifacts). When trying to supply 128 frames to P.js we simply don't get anything processed.
So, the question is if it would be possible to make P.js work with 128 frames? Currently I tried to convert the input of 128 frames to 4096 frames sized Float32Array but unfortunately all I could get is crackles.
The next step probably would be exploring ring buffers as explained here but I'm not sure if this the best option performance-wise. Or maybe there is really a simpler way to connect P.js with AudioWorklet and I'm simply missing something?
P.S.
Also I had problems running the Paulstretch.js from dist folder in AudioWorklet's enviroments, since we don't have access to window object so I convert it to ES6 module and it's easy to use it with Import and Export now. Let me know if you need this in case you will want to play and test P.js with AudioWorklet.
Hello @Catsvilles !
Normally the library is meant for being buffered, so you should be able to use the API of the paulstretch object to push small blocks of 128 sample until the buffer reaches the needed window size to give you an output block.
The trick is that the input array passed by the processor is erased and reused at every call of the function, so for each run you need to instantiate a new 128 samples array, copy the input to that array, and push that array to paulstretch with the write method.
I also realised that in this case, the API of the lib is not optimal because instantiating a new array is slow, and should be done as rarely as possible.
Can you send me your code ? I can try to make it work and improve the library upon it 😉 I'm also interested in the es6 refactor. You could make a pull request !
Thanks 🙏
@sebpiq Oof, I was afraid you will ask for this :) At this point I tried many different ways to make it work, the last way and the closest I got is trying to imitate the simple example you have with ScriptProcessor and Worker by using the AudioWorklet's message port but eventually do everything in AudioWorkletProcessor file, so currently the whole code is a complete mess and you probably will have a better chances using your own examples. I will attach the file anyway, but let me know if you cannot look at this and I will clean it up :)
Also I'm attaching the ES6 version, which is not pretty too, and was made for quick testing, but it does the job for now, eventually I will make it in a proper module and can PR if something.
As you can see I have this.frontBlocks = [] variable to push the processed blocks in, because if using only this.blocksOut we will get error when trying to process it with PaulStretch as it will never be enough of processed blocks. This comes from your simple example as you blocksOut array variable both in Worker and frontend, and eventually they have a bit different purpose, but maybe I misunderstood something