Distorted Sound when receiving recorded audioframe
winstondu opened this issue · 4 comments
In summary: Check out https://github.com/winstondu/Agora-RecordFrame-Bug. This project lightly modifies @CavanSu 's example code to add voice recording functionality, in order to demonstrate the issue I am talking about.
Background
Our app uses the chatRoomGaming Audio Profile in order to filter out background game sounds and other noises from the chat.
Currently, we want a method of recording the local user's voice (and only the local user's voice) when they are on a call. While we could obviously do so by tapping the unprocessed audio from iOS, we want to record the Agora-processed version of the local user's microphone audio that used Agora's chatRoomGaming audio profile filters.
In other words, we imagine the flow to be like this:
Unprocessed Mic Audio ----------(Agora Processing with profile) -------------> Processed Mic Audio (we want this) ---------> Agora Servers
I looked into the AudioFrameObserver api. In particular the onRecordAudioFrame function should have done the trick.
However, there is (what I believe to be) a bug: because I called setAudioProfile(.musicStandardStereo, scenario: .chatRoomGaming), the recorded voice is extremely distorted. It is a bug because if I never called the setAudioProfile command, the recorded audio is completely fine. In fact, if I change the setAudioProfile call even slightly, to setAudioProfile(**.speechStandard**, scenario: .chatRoomGaming), the recorded audio is fine as well.
The thing is, I can't not call setAudioProfile, since not calling it would mean I no longer benefit from the .chatRoomgGaming audio profile's background noise suppression on my Agora calls.
Sample Audio File.
In the following file, I recorded myself saying the words "testing, testing, testing", and this was the result:
IMG_1513.mp4.zip)
Smartphone (please complete the following information):
- Device: any iPhone
- OS: iOS 13.6
- Version: Agora iOS 3.0.1.1
@CavanSu , is there something I need to change to fix the voice distortion issue (seen in https://github.com/AgoraIO/Advanced-Audio/files/5127757/IMG_1513.mp4.zip)? Your Speech recognizer example only did remote audio, not the local recorded audio
Can you write audio data to pcm file directly? you upload mp4 file's sampleRate seems wrong @winstondu
@CavanSu , do you have a code snippet that illustrates how I can write to PCM file directly?
also, were you able to replicate my issue using my example project?
@CavanSu , I finally realized what the issue is.
For stereo channels, line 31 https://github.com/winstondu/Agora-RecordFrame-Bug/blob/e2269cd6bc05958441181faada72470d92aca8e3/SpeechRecognizer-iOS/MediaWorker.mm#L31
should have been
size_t size = audioFrame.bytesPerSample * audioFrame.samples * audioFrame.channels