Need help with settings

Question

Need help with settings

kafan1986 opened this issue 2 years ago · 6 comments

kafan1986 commented 2 years ago

I am trying to run two recorder in two separate thread parallelly. One recorder uses microphone and other uses the system audio. I have configured the both of these recorder to use 16000 sampling and 2 channels. I want these to record audio in chunks of say 3 seconds and then return the data for further processing. Below is the sample code.

The microphone is working as expected and returning after every 3 seconds with 48000 samples i.e. 16k * 3 but the issue is with the system audio (loopback mic) it returns frequently with 48000 samples like 3-4 times within a second. I am not sure why this weird behavior. I have also experimented by providing explicit block size value of say 128 but it seems to have no effect.

recorder = mic.recorder(samplerate=16000, channels=2)
max_num_frames = int(16000 * 3) # for 3 seconds

with recorder as record_object:
    data = record_object.record(max_num_frames)

Answer 1 · 2022-10-20T06:10:27.000Z

What sample rate are your devices using? On some operating systems, some audio devices do not honor the configured sample rate. (What OS are you using?)

Also, have you looked at the data? is it all-zeros? Some sound cards behave strangely when there's no audio data available. And this might be a bug in SoundCard, too, as it's not a behavior my sound cards seem to exhibit, so I can't test it.

Answer 2 · 2022-10-20T08:14:04.000Z

@bastibe UPDATE: There is something peculiar that I have observed. When some audio is playing at the background the system audio recorder works fine in the sense that it returns with samples after expected time period but if nothing is playing i.e. system audio is silent, the above issue reported is noticed, the data comes at a much higher frequency than theoretically possible. If I resume audio playback on system (play some youtube video etc.), the data rate again reverts back to normal till it paused again.

The OS is windows 11. I am still testing and will let you know. BTW, I am still unsure about block size and where it is used and is there a need to explicitly provide its value. Also, I have installed latest version from git master branch. Can I stop warning messages about data discontinuity?

Answer 3 · 2022-10-20T10:09:50.000Z

Can I stop warning messages about data discontinuity?

You can, using warnings.simplefilter. However, data discontinuity errors usually indicate that you didn't read the audio data fast enough, and that some audio data could not be recorded. They usually require you to fix the problem, not ignore the message.

Block sizes in this case refer to the number of frames you request per call to record. Request more frames per call to make things run more smoothly. This should fix your data discontinuity errors as well. A data discontinuity happens if you don't read data fast enough, i.e. don't call record often enough, or don't record enough frames per call.

(But please let me know if you think that's wrong. These warnings are a very new feature and I don't have much experience with Windows' behavior in these cases, yet.)

So the frame spam only happens during silence. In that case, Windows does not actually return any audio data, but instead sets a flag that says there's only silence. But according to the documentation and example code, the returned number of frames should still be valid. Yet in your case, this does not seem to be the case. How very annoying.

Honestly, I'm not sure what to do in this case.

Answer 4 · 2022-10-20T10:25:55.000Z

Can I stop warning messages about data discontinuity?

You can, using warnings.simplefilter. However, data discontinuity errors usually indicate that you didn't read the audio data fast enough, and that some audio data could not be recorded. They usually require you to fix the problem, not ignore the message.

Block sizes in this case refer to the number of frames you request per call to record. Request more frames per call to make things run more smoothly. This should fix your data discontinuity errors as well. A data discontinuity happens if you don't read data fast enough, i.e. don't call record often enough, or don't record enough frames per call.

(But please let me know if you think that's wrong. These warnings are a very new feature and I don't have much experience with Windows' behavior in these cases, yet.)

So the frame spam only happens during silence. In that case, Windows does not actually return any audio data, but instead sets a flag that says there's only silence. But according to the documentation and example code, the returned number of frames should still be valid. Yet in your case, this does not seem to be the case. How very annoying.

Honestly, I'm not sure what to do in this case.

Is this flag visible to us through your library? I can think of some logic to handle or skip data processing at my end with such silence frame data.

Answer 5 · 2022-10-20T10:49:09.000Z

You are recording from 1. Microphone; 2. Speaker loopback; Right? What is your speaker playing so you can record its loopback? If I understood correctly, your Microphone is recording about 16k samples per second, but your loopback is recording about 48k samples per second. It seems that you are playing from an outside application, e.g. Spotify. So the Windows audio stack probably locked the samplerate of the output device to whatever samplerate is set in the system, or the outside application, then the samplerate of the loopback From the speaker is also locked. Test if this unexpected behaviour also happens if you are playing from your Python application. This could give a bit more insight on where the problem is relying, system or library.

Answer 6 · 2022-10-20T10:56:00.000Z

Is this flag visible to us through your library? I can think of some logic to handle or skip data processing at my end with such silence frame data.

No, because according to the documentation, it merely means that the recorded data ought to be all-zeros. Thus it means either SoundCard is handling the condition incorrectly somehow, or there's a bug in how Windows presents (or documents) the flag.