Strange error occurs when recording a lot of audio in a session (Firefox)

[mylnikovD]

Hi Chris, thanks for amazing library, everything seem to work great, but lately I've started to encounter the following issue

It appears sometimes as recording not showing, and sometimes it has duration of 0.
We didn't test it thoroughly on Chrome, however, we've met a similar situation, when after firing a 'stop' event, the audio didn't appear. Memory in browser didn't seem very high, 25MB on a tab with media recorder. I'm using it in React, with wavesurfer.js in a following way:

// in App.js
import { register } from 'extendable-media-recorder';
import { connect } from 'extendable-media-recorder-wav-encoder';
useEffect(() => {
    async function init() {
      await register(await connect());
    }
    init();
  }, []);
// usage
this.waveSurfer = WaveSurfer.create({
          container: '#waveform',
          height: 38,
          waveColor: '#666666',
          barWidth: 1,
          interact: false,
          barHeight: 34,
          plugins: [MicrophonePlugin.create({ bufferSize: 2048 })],
        });
        const audioChunks: any[] = [];
        this.waveSurfer.microphone.on('deviceReady', stream => {
          this.mediaRecorder = new MediaRecorder(stream, { mimeType: 'audio/wav' });
          console.log('In deviceReady (mediaRecorder)', this.mediaRecorder);
          if (!this.mediaRecorder) return;
          this.mediaRecorder.start();
          this.mediaRecorder.addEventListener('dataavailable', (event: any) => {
            console.log('In dataavailable (event data)', event.data);
            audioChunks.push(event.data);
          });

          this.mediaRecorder.addEventListener(
            'stop',
            () => {
              try {
                const audioBlob = new Blob(audioChunks, { type: 'audio/wav' });
                const audioUrl = URL.createObjectURL(audioBlob);
                this.addNewRecord({
                  id: this.state.recordedSamples.length,
                  audioUrl,
                  selected: false,
                  blob: audioBlob,
                });
              } catch (error) {
                console.error('Error in stop handler', error);
              }
            },
            { passive: true }
          );
        });
        this.waveSurfer.microphone.start();

Also, there's this console message when recording the audio, not sure if it's related to your package or wavesurfer.js, but maybe it'll help

[chrisguttandin]

Hi Dmitry,

thanks for reporting this. Unfortunately I can't reproduce it locally. What exactly does "a lot of audio" mean? Are there some conditions which make it always fail?

I noticed that you're example is missing the MediaRecorder import. But I guess that's only a copy/paste error since the stack trace looks like it's imported correctly.

The error actually comes from deep inside the AudioWorkletProcessor and indicates that it somehow received an unexpected sequence of messages.

https://github.com/chrisguttandin/recorder-audio-worklet-processor/blob/master/src/recorder-audio-worklet-processor.ts#L106

I would really like to fix this bug but there is something which could help you in the short term. When calling mediaRecorder.start() without a timeslice parameter the 'dataavailable' event will only fire once with the entire blob. If you specify the timeslice parameter the 'dataavailable' event will fire whenever a slice of the final output is ready. At least in theory this should make calling stop() faster. It's not noticeable for short recordings but does make quite a difference for recordings of a couple of minutes.

[chrisguttandin]

I think the console warning is unrelated. It hopefully goes away once this fix (chrisguttandin/standardized-audio-context#982) is published on npm.

[mylnikovD]

Hi Chris, thanks for advice, gonna try it shortly and hopefully it helps.
By recording a lot of audio I meant that this code under usage is being invoked in componentDidUpdate of my recording component, which is triggered by changing the state after clicking on "record" button. When user clicks it again, I'm calling this.mediaRecorder.stop(), and then the cycle begins anew when user clicks "record" again.
I thought the problem may be connected to rewriting some values to this.mediaRecorder or this.wavesurfer that happens in this cycle, but refreshing the page should help in this case, and it sometimes does, but sometimes it doesn't.

[chrisguttandin]

Do I understand that correctly that you basically call start() directly after stop()? Somehow like this:

first click:
  start

second click:
  stop
  start

third click:
  stop
  start

...

[mylnikovD]

no, each click toggles recording, so it's one click - one start/stop

[chrisguttandin]

Hi Dmitry, I came up with this little demo which surprisingly breaks in Chrome but not in Firefox.

import { MediaRecorder, register } from 'extendable-media-recorder';
import { connect } from 'extendable-media-recorder-wav-encoder';

connect()
    .then((port) => register(port))
    .then(() => {
        window.onclick = () => {
            window.onclick = null;

            navigator.mediaDevices.getUserMedia({ audio: true })
                .then((stream) => {
                    const mediaRecorder = new MediaRecorder(stream, { mimeType: 'audio/wav' });

                    const audioChunks: Blob[] = [];

                    mediaRecorder.start();
                    mediaRecorder.addEventListener('dataavailable', ({ data }) => audioChunks.push(data));
                    mediaRecorder.addEventListener('stop', () => {
                        const blob = new Blob(audioChunks, { type: 'audio/wav' });

                        audioChunks.length = 0;

                        console.log(blob);
                    });

                    window.onclick = () => {
                        mediaRecorder.stop();
                        mediaRecorder.start();
                    };
                });
        };
    });

Could you please tweak it as needed to reproduce your use case? I meanwhile try fixing it in Chrome.

[mylnikovD]

Seems that everything copies my behavior, if we remove react and wavesurfer wrappers, so you did great!
This bug is kinda ghostly, it's quite rare to be honest, but if you managed to see something wrong in Chrome, I assume you must be close to resolve this mystery :)
On chrome we encounter other bug which sets microphone volume to max on its own, but as far as I researched, it's not related to your library, but a chrome itself bug which is fixed by some extensions,
https://support.google.com/chrome/thread/7542181/chrome-is-auto-adjusting-the-microphone-level?hl=en
But this extension fix doesn't help to one computer we've been testing the original bug, so we couldn't test it properly

[chrisguttandin]

I think I found the problem in Firefox. There was a race condition which caused the AudioContext that gets used internally to get suspended even though a recording was underway. This should be fixed now.

I also made an update to the implementation used in Chrome which fixes a similar issue. 148a769

I think there is nothing we could change in this library to fix the microphone level bug you mentioned. But please let me know if you think that' not the case.

Thanks again for reporting this bug.

[mylnikovD]

it seems that this issue is actually gone, thanks so much for your amazing job!

[chrisguttandin]

Thanks for letting me know. I'm happy to hear that.