Firefox-only issue! "Uncaught (in promise) Error: The internal state does not allow to process the given message."
Closed this issue · 3 comments
Hey there! 👋
I've just started using your library for my final year college project (which can be found here. It works brilliantly for recording the output of Tone.js (a Web Audio library I'm using) and encoding as lossless WAV instead of lossy OGG/WEBM. To do this I'm also using extendable-media-recorder-wav-encoder.
I've been developing this app using Chrome for the most part, as the V8 engine helps a lot with performance, even though Firefox is my browser of choice. While encoding as WAV works perfectly on Chrome, it is a bit weird on Firefox.
After importing and running await register(await connect()) I initialise the extendable MediaRecoder like so:
const REC = new MediaRecorder(REC_DEST.stream, { mimeType: 'audio/wav' });REC_DEST being a media stream destination connected to Tone.js' own Web Audio context:
const REC_DEST = Tone.context.createMediaStreamDestination()Where the code breaks is between starting and stopping the recording on Firefox, and weirdly enough it doesn't always break. The first few recordings usually work fine, and then occasionally it fails to stop and save the recording, throwing this error message:
Uncaught (in promise) Error: The internal state does not allow to process the given message.
createListener listener.ts:19
listener on.ts:6
createOn on.ts:8
subscribe wrap-subscribe-function.ts:10
createRecorderAudioWorkletNodeFactory recorder-audio-worklet-node-factory.ts:38
createPromisedAudioNodesEncoderIdAndPort web-audio-media-recorder.ts:35
promisedAudioNodesAndEncoderId web-audio-media-recorder.ts:193
promise callback*start web-audio-media-recorder.ts:192
start media-recorder-constructor.ts:220
startRecording synth.js:1470
<anonymous> synth.js:1519
sendEvent webaudio-controls.js:178
pointerdown webaudio-controls.js:1362
WebAudioControlsWidget webaudio-controls.js:113
WebAudioSwitch webaudio-controls.js:1148
CustomElementConstructor* webaudio-controls.js:1146
The error only ever occurs after I've toggled the recording button in my app to off, which calls REC.stop() before REC.ondataavailable() and REC.onstop() are called to push the audio stream into a chunks array, before creating an audio/wav Blob.
After using the debugger, it seems to happen between REC.start() and REC.stop() but the error prints to the console after REC.stop() is called. I just can't tell where it's coming from exactly.
I tried making my functions async/await and adding .then() and .catch() blocks, as per a StackOverflow suggestion regarding that error message, but that didn't help. I also tried creating a new MediaRecorder every time I toggle the recording button on, just before calling REC.start() as I saw that suggested in an old issue, but that didn't help either. The only thing that did help was conditionally changing the mimeType to 'audio/webm;codecs=opus' when Firefox is being used.
Does Firefox just not support WAV? I know it doesn't with native MediaRecorder, as logging MediaRecorder.isTypeSupported('audio/wav') to the console on Firefox returns false by default, but while using your library it returns true.
For now, conditionally changing the mimeType works fine, and I can just warn users that using Firefox will limit them
to lossy recording, but I just wanted to see if anyone has any suggestions as to why this is happening...
This is my first issue submission, so apologies if it's lacking information. Am happy to provide more info if need be. You can check the repository I linked if you want a look at the entire script, this is all occurring in src/synth.js.
Here's a link to the REC.start() line from the commit before the conditional fix.
Here's a link to the REC.start() line from the conditional fix commit (REC.start() is now in its own async function, but this made no difference, the only functional difference is the if/else defining the mimeType depending on the browser being used)
Thanks in advance! <3
Hi @Mangoshi,
thanks for submitting this issue. It's definitely an interesting one. :-)
extendable-media-recorder somehow needs to get the uncompressed audio data to support arbitrary codecs. Usually that would be a task for the Web Audio API but there is a bug in Chrome which makes it unusable for that use case. That's the reason why a recording in Chrome works quite differently as in all other browsers right now.
When you record something in Firefox the Web Audio API is used under the hood to do the recording. extendable-media-recorder depends on standardized-audio-context to handle some cross browser quirks of the Web Audio API. And that's where it gets interesting. Tone.js uses standardized-audio-context, too.
As the author of standardized-audio-context I can assure you that I took great care to not patch any global objects to make it possible to use many instances or even versions of standardized-audio-context in parallel. But you never know.
But there is also a chance for a race condition somewhere in the WebAudioMediaRecorder. That's the internal recorder that uses the Web Audio API. Do you always wait for the 'stop' event being triggered by the previous recording before you start the next one? There seems to be nothing in your code which checks that but maybe you are doing this manually.
By the way, thanks a lot for all the details. I wish all bug reports were like this.
Hey @chrisguttandin, thanks for the speedy response!
Hmm, that is interesting considering how you wrote standardized-audio-context...
It's cool to know Tone uses it too, but extra weird that it's giving me this issue in that case!
I tried checking the state of the MediaRecorder before running .stop() and .start() like so:
async function startRecording() {
// log state of MediaRecorder
console.log("MediaRecorder state (pre-start):", REC.state)
if(REC.state === "inactive"){
// clear chunks
CHUNKS = []
// start recording
await REC.start()
// log state of MediaRecorder
console.log("MediaRecorder state (post-start):", REC.state)
// set label to "recording..."
recorderLabel.innerHTML = "Recording..."
} else {
console.log("MediaRecorder is not inactive!")
}
}async function stopRecording() {
// log state of MediaRecorder
console.log("MediaRecorder state (pre-stop):", REC.state)
if (REC.state === "recording") {
// stop recording
await REC.stop()
// log state of MediaRecorder
console.log("MediaRecorder state (post-stop):", REC.state)
// return label to default
recorderLabel.innerHTML = "Record"
} else {
console.log("MediaRecorder is not recording!")
}
}But the same issue occurs sadly. Is this logic enough to ensure stop has always been triggered before starting the next one? I wasn't doing this before because the toggle logic seemed good enough; If the button's state is on (1), it will only ever fire .stop() if it's pressed again, and visa versa.
Also, I'm glad the details I provided were appreciated!
These issues are so context dependant, so I wanted to be sure you had as much as possible :)
I think your toggle logic is enough and it should work without waiting for any events. I was just curious in which situations the bug occurs and if there is any specific behavior to narrow it down.
I think I found the issue. Version 7.1.10 should have the fix.
The error message that you were getting was actually coming from here: https://github.com/chrisguttandin/recorder-audio-worklet-processor/blob/ee9eeea42551a2ca2942baccc882efbc3805c5f3/src/recorder-audio-worklet-processor.ts#L106
It's the AudioWorkletProcessor that records the audio. And for some reason it ended up in an undefined state. When looking at the code again I noticed that there was a chance that the recorder stopped itself when it only received silence. There is no way to communicate that back to the code which initiated the recording and consequently it sometimes tried to stop the already stopped recorder. This triggered the error.
Thanks again for your detailed bug report which really helped to solve this.