Audio doesn't record with ARKit RealityKit ARView

[burgalon]

Hey Vlad,
Thank you for this package!

The video recording seems to be working well, however I was not able to make the audio recording work with ARKit ARView.
I've added the custom ARView+MetalRecordable.swift and ARView+SelfSceneRecordable.swift
and set configuration.providesAudioData = true

Is this a known issue with ARView or maybe I'm missing something?

Thanks,
Alon

[furysheep]

@burgalon
I faced a similar problem. When I set providesAudioData to true, the ARSession was failed to start. I needed to use AVAudioSession separately and need to combine it with the video frames from ARKit.

[v-grigoriev]

Hello guys.
Look at RealityKitViewController in the example app. (oh, I seem to have a wrong link in the readme.)
For me, it records audio as well as video, but there is something wrong with the AR configuration since I can't see any anchors.

To enable the controller in the example app, comment out #if !COCOAPODS and #endif in MainViewController in seems I made a mistake in the condition and it's never being shown.

And of course, you can always use AVAudioSession, I hope that with a helper I built it's pretty straightforward try? .makeAudioForRecorder(sceneView.recorder!)

[burgalon]

Thank you for the reply.
I was just able to solve this by delegating the didOutputAudioSampleBuffer like this in my ARView:

  @objc public func session(
    _ session: ARSession,
    didOutputAudioSampleBuffer audioSampleBuffer: CMSampleBuffer
  ) {
    self.sceneRecorder!.session(session, didOutputAudioSampleBuffer: audioSampleBuffer)
  }

For some reason I thought this delegation was taken care of by SceneRecorder...

[v-grigoriev]

Yea, you are right.
In my example, I do the same by setting session.delegate = sceneRecorder

Since SCNRecorder doesn't have access to ARSession directly in case of ARView I can't configure it by default. As I do for ARSCNView.