AudioUnit hell

Question

AudioUnit hell

LonestarX91 opened this issue 4 years ago · 11 comments

Hello,

So i have a big issue. The story starts like this: I need audio only (no video), a few days ago I tried using https://github.com/miche-atucha/taresip which was lovely managed to call, could hear the other client (even tho it had a small issue of unintentional audio passthrough, it was pretty much ok). ALSO, IT USED COREAUDIO surprised pikachu. As far as I looked, that was built with baresip-0.4.20, re-0.4.17 and rem-0.4.7.

Now here comes problem#1: I needed TLS. So i found this https://github.com/rtcexpert/baresip-ios-bitcode (bless his soul), which is also a pending PR here and it connects fine. So i took the mk, i grabbed baresip 0.6.6, re 0.6.1 and rem 0.6.0, openssl&co, built it just fine. problem is if i wanna add coreaudio module in

BARESIP_BUILD_FLAGS_X86_64 := \
	$(BARESIP_BUILD_FLAGS) \
	EXTRA_MODULES='g711 coreaudio srtp audiounit avcapture opengles'

it will fail with a metric ton of errors like here (baresip/baresip#593). So I did what @eriksundin suggested and patched coreaudio.c with

CFStringRef coreaudio_get_device_uid(const char *name)
{
	#if TARGET_OS_IPHONE
		return NULL;
	# else
        ....
       #endif
}

and then i got some more errors like

CC [m]  /Users/dani/Downloads/baresip-ios-bitcode-master/build/aarch64/baresip/modules/coreaudio/recorder.o
modules/coreaudio/recorder.c:78:51: error: too many arguments to function call, expected 2, have 3
        rh(inQB->mAudioData, inQB->mAudioDataByteSize/2, arg);

So I just gave up...

Now you ask what happens if i just use the default audiounit ? I get the following logs

2020-07-15 19:34:47.336010+0300 baresiptest[1728:295101] uaInitErr: 0
aucodec: PCMU/8000/1
aucodec: PCMA/8000/1
auplay: audiounit
ausrc: audiounit
medianat: stun
medianat: turn
medianat: ice
medianat: ice-lite
Populated 2 audio codecs
Populated 0 audio filters
Populated 0 video codecs
Populated 0 video filters
2020-07-15 19:34:47.378361+0300 baresiptest[1728:295101] confModulesErr: 0
2020-07-15 19:34:47.378498+0300 baresiptest[1728:295101] uaAllocErr: 0
2020-07-15 19:34:47.378883+0300 baresiptest[1728:295101] REGISTERED: YESSSSS
ua: using best effort AF: af=AF_INET
...........connecting.................
call: answering call from sip:2000@xxx with 200
stream: update 'audio'
audio: Set audio decoder: PCMU 8000Hz 1ch
audiounit: AudioRouteChange - reason 3
audio: player started with sample format S16LE
audio: Set audio encoder: PCMU 8000Hz 1ch
audiounit: record: enable resampler 8000.0 -> 8000 Hz
2020-07-15 19:34:49.054507+0300 baresiptest[1728:295140] [aid] AudioIssueDetectorNode.cpp:166:Initialize: Caught analyzer graph exception 1718775073 !mrf slice duration must be set before configure. in /Library/Caches/com.apple.xbs/Sources/AudioDSP/AudioDSP-366.38/CoreAudioUtility/Source/CADSP/DSPGraph/DSPGraph_Graph.cpp:628
audio: source started with sample format S16LE
audio tx pipeline:   audiounit ---> PCMU
audio rx pipeline:   audiounit <--- PCMU
�[31mmain: long async blocking: 656>100 ms (h=0x1046e3f48 arg=0x28152cbf0)
�[;mcall: SIP Progress: 180 Ringing (/)
call: SIP Progress: 180 Ringing (/)
2020-07-15 19:34:49.089235+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.112152+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.135369+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.158808+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.182018+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2000@xxx: Call established: sip:2000@xxx
2020-07-15 19:34:49.205226+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.228837+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.252263+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.275400+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.299691+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
stream: update 'audio'
audio: Set audio decoder: PCMU 8000Hz 1ch
2020-07-15 19:34:49.322460+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.344792+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
audio: player started with sample format S16LE
audio: Set audio encoder: PCMU 8000Hz 1ch
2020-07-15 19:34:49.682740+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.817741+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.821564+0300 baresiptest[1728:295140] [aurioc] AURemoteIO.cpp:1086:Initialize: failed: -66635 (enable 3, outf< 1 ch,      0 Hz, Float32> inf< 1 ch,   8000 Hz, Int16>)
�[31maudiounit: record failed: -66635 (\377\376\373\265)
�[;m�[31maudio: start_source failed (audiounit.default): Operation not supported by device
�[;msip:2000@xxx:5061;transport=tls: session closed: Operation not supported by device
2020-07-15 19:34:49.840826+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
�[31mmain: long async blocking: 531>100 ms (h=0x1046e3f48 arg=0x28152cbf0)
�[;m2020-07-15 19:34:49.864077+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.887167+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.910383+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.933624+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.956830+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.980111+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:50.003410+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:50.026485+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:50.049776+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:50.073054+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:50.096461+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:50.119868+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:50.143138+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:50.166310+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:50.189574+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:50.212550+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:50.236056+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
sip:2000@xxx: session closed: Connection reset by peer
2020-07-15 19:34:50.258984+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
sip:2000-0x12de04f80@192.168.0.144:57128;transport=tls: Call with sip:2000@xxx terminated (duration: 1 sec)

I tried multiple configs such as:
https://gist.github.com/LonestarX91/ff4922fd03b9aa88a1e8dbbb79d78a32
https://gist.github.com/LonestarX91/483bfeb7327af18e5776dcf084d47770
same result
And obviously... no audio.

If anyone can please help me get either (coreaudio or audiounit) working, i'll raise them a statue

THANKS

Answer 1 · 2020-07-15T23:47:52.000Z

Switched from that fork to baresip-ios repo, patched contrib.mk for TLS (https://gist.github.com/LonestarX91/e5aca98ab051999499e7b59ec18892b7) and built for 0.6.0 all (which was HEAD). still getting those pesky errors you see, but I also discovered this which seems to encounter my issue baresip/baresip#784... You said you pushed it to master but I don't understand how it still happens... :/

Answer 2 · 2020-07-16T00:17:54.000Z

switched to 0.6.6 baresip, same problem
log: https://gist.github.com/LonestarX91/b2974ae949ebb131964ee8ec651533ba
config: https://gist.github.com/LonestarX91/447b3e9ff0883ab98027ec6e9a68628b
hopes and dreams:

Answer 3 · 2020-07-16T10:35:47.000Z

Well, finally ended my story with patching baresip 0.6.1 coreaudio with this ylevytskyy/baresip@66b1473

Now audio works fine and I have TLS. Would've been nice to use audiounit but beggars can't be pickers

hope this thread helps anyone

Answer 4 · 2020-08-20T13:23:22.000Z

@LonestarX91

Can you describe better how you solved the audio problem? I have the same problem:

2020-08-20 10: 16: 09.913201-0300 APP [652: 138895] [aid] AudioIssueDetectorNode.cpp: 166: Initialize: Caught analyzer graph exception 1718775073! Mrf slice duration must be set before configure. in /Library/Caches/com.apple.xbs/Sources/AudioDSP/AudioDSP-366.38/CoreAudioUtility/Source/CADSP/DSPGraph/DSPGraph_Graph.cpp:628
2020-08-20 10: 16: 09.934585-0300 APP [652: 138895] [aurioc] AURemoteIO.cpp: 1590: Start: AUIOClient_StartIO failed (-66637)

Answer 5 · 2020-11-06T13:06:11.000Z

Got the same error:
AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""

with current master version with audiounit configured, without coreaudio.

[0:00:16] audio=63467/63467 (bit/s)    
2020-11-06 14:47:40.231683+0200 AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-11-06 14:47:40.256133+0200 AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-11-06 14:47:40.279915+0200 AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-11-06 14:47:40.302482+0200 AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-11-06 14:47:40.325733+0200 AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""

Answer 6 · 2020-11-09T15:45:31.000Z

And probably with open ssl:

AudioIssueDetectorNode.cpp:166:Initialize: Caught analyzer graph exception 1718775073 !mrf slice duration must be set before configure. in /Library/Caches/com.apple.xbs/Sources/AudioDSP/AudioDSP-366.38/CoreAudioUtility/Source/CADSP/DSPGraph/DSPGraph_Graph.cpp:628

Answer 7 · 2020-11-10T16:20:22.000Z

Confirm the errors like:

AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""

but can hear the audio on iOS. Stack:

audiounit
srtp (with openssl)
opus (or without)

Answer 8 · 2020-12-30T16:56:41.000Z

@Koshub did you find any solution? I am using Xcode11 and Xcode10 with ios 13 and ios 14, I am not getting audio...

Answer 9 · 2020-12-30T18:32:40.000Z

@samirmagnates answered here

Answer 10 · 2021-01-03T14:44:00.000Z

please create a new ticket for this, use latest versions from git HEAD.
please provide exact info about software/hardware etc.

Answer 11 · 2021-05-13T15:01:23.000Z

https://github.com/robovm/apple-ios-samples/blob/master/CoreAudioUtilityClasses/CoreAudio/AudioUnits/AUPublic/Utility/AUBuffer.h
I tracked down the source which logging it but cant turn it off. For my case, it work normally except annoying message.