MacOS: Empty result for all_speakers/all_microphones
allquixotic opened this issue · 28 comments
Symptom:
% ptpython
>>> sys.version
'3.10.8 (main, Oct 21 2022, 22:22:30) [Clang 14.0.0 (clang-1400.0.29.202)]'
>>> import soundcard as sc
>>> sc.all_speakers()
[]
>>> sc.all_microphones()
[]
>>> sc.default_speaker()
<Speaker HyperX Cloud Alpha Wireless (0 channels)>
>>> sc.default_microphone()
<Microphone HyperX Cloud Alpha Wireless (0 channels)>
Note all_speakers() and all_microphones() return empty lists instead of the expected data. The other strangeness is that it says the default speaker and microphone have "0" channels.
I have Python 3.10.8 installed on MacOS Ventura using Homebrew. To rule out environment problems, I've checked env for any weird variables, cleared out my site-packages directory and reinstalled soundcard and all its dependencies, but the problem persists.
Just the other day, all of this was working as expected, but after doing a complete reinstall of Python and soundcard, this no longer works.
As a proof that my CoreAudio native libraries are working, the program SwitchAudio (installed by brew install switchaudio-osx; source code at https://github.com/deweller/switchaudio-osx ) returns the following output showing all my devices:
% SwitchAudioSource -a -f json
{"name": "HyperX Cloud Alpha Wireless", "type": "input", "id": "51", "uid": ""}
{"name": "MacBook Pro Microphone", "type": "input", "id": "148", "uid": ""}
{"name": "Loopback Audio", "type": "input", "id": "153", "uid": ""}
{"name": "HyperX Cloud Alpha Wireless", "type": "output", "id": "51", "uid": ""}
{"name": "MacBook Pro Speakers", "type": "output", "id": "141", "uid": ""}
{"name": "Loopback Audio", "type": "output", "id": "153", "uid": ""}
Technically, I could drop the soundcard dependency, call switchaudio-osx via the CLI, parse the resulting JSON, and get what I need. But I wanted to report this issue in the soundcard lib.
Looking at the definition of all_speakers() for CoreAudio in soundcard/coreaudio.py:
def all_speakers():
"""A list of all connected speakers."""
device_ids = _CoreAudio.get_property(
_cac.kAudioObjectSystemObject,
_cac.kAudioHardwarePropertyDevices,
"AudioObjectID")
return [_Speaker(id=d) for d in device_ids
if _Speaker(id=d).channels > 0]
See how it only creates a Speaker object if channels > 0 ? Well, the default speaker and default microphone are showing up as channels = 0, for some reason, so maybe all of my devices are showing up as channels = 0 using the API...
Based on the native code here (which we know works, because switchaudio-osx works on my system), it looks like the soundcard CFFI code is probably correctly enumerating all the devices. Unfortunately, switchaudio doesn't implement channel detection, so I can't vouch for the correctness of this code in soundcard/coreaudio.py:
@property
def channels(self):
bufferlist = _CoreAudio.get_property(
self._id,
_cac.kAudioDevicePropertyStreamConfiguration,
'AudioBufferList', scope=_cac.kAudioObjectPropertyScopeOutput)
if bufferlist and bufferlist[0].mNumberBuffers > 0:
return bufferlist[0].mBuffers[0].mNumberChannels
else:
return 0
Looks like it gets to the return 0 there at the end, which is the source of all the problems... unfortunately I don't know why this happens, or how to fix it.
Thank you for the bug report.
Could you try to run something like
"""A list of all connected speakers."""
device_ids = _CoreAudio.get_property(
_cac.kAudioObjectSystemObject,
_cac.kAudioHardwarePropertyDevices,
"AudioObjectID")
return [_Speaker(id=d) for d in device_ids]
This should confirm whether the channel detection is somehow broken, or the problem is elsewhere.
Also, could you grab the .channels property of your default_speaker()?
>>> import soundcard
>>> device_ids = soundcard.coreaudio._CoreAudio.get_property(soundcard.coreaudio._cac.kAudioObjectSystemObject, soundc
... ard.coreaudio._cac.kAudioHardwarePropertyDevices, "AudioObjectID")
>>> [soundcard.coreaudio._Speaker(id=d) for d in device_ids]
[<Speaker P27h-20 (0 channels)>, <Speaker C49RG9x (0 channels)>, <Speaker C49RG9x (0 channels)>, <Speaker HyperX Cloud Alpha Wireless (0 channels)>, <Speaker Shure MV88+ (0 channels)>, <Speaker MacBook Pro Microphone (0 channels)>, <Speaker MacBook Pro Speakers (0 channels)>, <Speaker Microsoft Teams Audio (0 channels)>, <Speaker Loopback Audio (0 channels)>]
soundcard.default_speaker().channels is also 0.
In short: yes, everything is showing as 0 channels.
I ended up bypassing the need for the Python soundcard lib by rewriting my code in Rust and using a CoreAudio binding to do what I needed (after I learned how the heck to use CoreAudio in the first place);
https://github.com/allquixotic/macpitch/blob/main/src/main.rs
I don't actually read or use the channels in any of this, but I'm more than willing to try out code in either Python or Rust that calls CoreAudio on my system and attempts to identify the number of channels for my devices.
Just as a reminder, this is on MacOS Ventura (latest stable version) on a M1 Max MacBook Pro (latest MBP model) that this seems to go awry. I also have several external displays and USB soundcards attached.
I'm not sure what exactly caused all the channels to start displaying as 0. I also have the Rogue Amoeba Audio Capture Engine (ACE) installed, which is a kernel-mode driver that provides "loopback" functionality for CoreAudio devices. I'm not sure how this could/would affect the number of channels in the CoreAudio API, though.
This is the first time I've heard of a bug like this. Thus it might either be something very new, or something specific to your machine.
Could this be in some way related to a new macOS security measure? (It wouldn't be the first time). For example, are you using a third-party terminal application, or are you calling the code from an untrusted application? Some applications need to be whitelisted in accessibility settings before they are allowed to work correctly.
It also wouldn't be the first time for macOS to change some behavior of the C API "silently". Do you know whether this might have happened in response to some update you installed?
And, if you would indulge me, could you try testing the number of channels in your Rust wrapper? I wonder if this is somehow specific to soundcard, or in fact applies to every CoreAudio application.
If you don't have time to help me here, that's perfectly understandable as well. But I'd be grateful for your help, as I personally don't have a recent macOS machine to test these things on!
@bastibe Slightly off topic as I don't wish to create a fresh ticket. I have an older intel based macbook air and the system is able to take audio feed from microphone and lists all microphone connected but it does not detect any system audio i.e. loopback=True. Any idea why this could be?
You should have seen a warning saying that the loopback functionality is not available on macOS. If you know how to implement it on macOS, I'd be grateful for a hint, or pull request.
Blackhole project is another project which achieves this, although it is a different product suite in itself.
I'd never heard of Blackhole, but seemingly it's very similar to Rogue Amoeba's Audio Capture Engine, except that it's open source (GPL3). Embedding Blackhole within Python SoundCard is impractical, because there are installation steps to installing the Blackhole package at the system level, which we can't coordinate when just installing a Python package.
Also, licensing: because this project is licensed under the BSD license, which prevents combining this code with GPL3, it would probably violate the GPL to automatically install Blackhole with SoundCard.
I'm not sure there's a better way to do it than to instruct Mac users who need loopback to install Blackhole or Audio Capture Engine (or equivalent; there might be other competitors).
We can't support the SoundCard loopback property on any particular sound device, though, because the name of any given audio capture device created by Blackhole, ACE, etc. could be user-defined, so we have no way of identifying which device might be a loopback.
Presumably, there's some way of implementing loopback devices on macOS within soundcard. I just haven't found the relevant API in Core Audio, yet.
Loopback devices were a later addition to soundcard, partly contributed by pull requests. They were not included in the original release, and therefore were not part of my original research. Which is to say, there might be a trivial way to do this.
@bastibe @allquixotic There is another similar package https://github.com/kyleneideck/BackgroundMusic#recording-system-audio that enables system audio recording. Maybe worth checking it out if not done already.
This is the first time I've heard of a bug like this. Thus it might either be something very new, or something specific to your machine.
I have a few other Macs floating around that I'll test on when I get a chance, but it may not be soon.
Could this be in some way related to a new macOS security measure? (It wouldn't be the first time). For example, are you using a third-party terminal application, or are you calling the code from an untrusted application? Some applications need to be whitelisted in accessibility settings before they are allowed to work correctly.
Maybe, but I tried it both with iTerm2 and the native Mac Terminal app, and neither one pops up a security prompt asking me to permit microphone access or anything like that. They don't appear as options in Privacy to even enable microphone access, because they don't seem to need it.
It also wouldn't be the first time for macOS to change some behavior of the C API "silently". Do you know whether this might have happened in response to some update you installed?
The only possible update it could have been is between Ventura 13.0.0 and Ventura 13.0.1. I'm not 100% sure I had tested this both before and after the 13.0.1 upgrade, as I wasn't paying attention to specifically when I installed the update vs. when I was messing with python-soundcard.
And, if you would indulge me, could you try testing the number of channels in your Rust wrapper? I wonder if this is somehow specific to soundcard, or in fact applies to every CoreAudio application.
I wasn't able to get this working in Rust, but that's mainly due to my own incompetence with figuring out how to do raw memory allocation in Rust. The furthest I got was some program crashes. I can't put any more time into it unfortunately.
If you don't have time to help me here, that's perfectly understandable as well. But I'd be grateful for your help, as I personally don't have a recent macOS machine to test these things on!
@bastibe I can repro this issue on my M1 MacBook Pro running Ventura 13.1 on System python (3.9.6) but not on Homebrew installed python (3.10.4). Let me know if I can help troubleshoot
I bet that's a sandboxing issue. In the past, some people needed to activate something in the accessibility settings to get around issues like this. As far as I know, there is nothing I can do about that in the software itself.
If you do find a reason or workaround, please let me know.
I have two Macs where I have tested the code
- Intel Mac with macOS Monterrey soundcard is working
- Soundcard 0.4.2
- Intel Mac with macOS Ventura 13.4 soundcard is working
- Soundcard 0.4.2
- M2 Pro with Ventura 13.3.1 same issue channels not appearing
- Soundcard 0..4.2
- M2 Pro with Ventura 13.4 same issue channels not appearing
- Soundcard 0.4.2
I’m updating the Intel Mac to see if it’s related to Apple Silicon or Ventura.
I use an app to request permission for the terminal so that’s not the reason
Edited: I'm using a Focusrite 3th gen
Edited2:
I have finished updating the Intel Mac with Ventura 13.4 and SoundCard still works.
@allquixotic One question the setup work at some point on the Apple silicon Mac?
I think I found the problem @bastibe
The channel number is not automatically assigned on Apple Silicon
I'm using this Function from Coreaudio
Apple Silicon
AudioChannelDescription[] 1
0.0
Intel
AudioChannelDescription[] 1
1.401298464324817e-45
I found that it's possible to use this
AudioChannelDescription *channelDescription = [[AudioChannelDescription alloc] init];
[channelDescription setChannelLabel:@"Left" forChannelNumber:0];
I'm not too experienced with CoreAudio so I'll prefer suggestions on this
Please let me know if this makes sense
More interesting findings with this issue. I wanted to verify that the Permission theory was correct.
So I wrote this CPP code
#include <CoreAudio/CoreAudio.h>
#include <AudioUnit/AudioUnit.h>
void requestMicrophonePermission(AudioDeviceID deviceID) {
OSStatus status;
AudioObjectPropertyAddress selectorAddress;
selectorAddress.mSelector = kAudioDevicePropertyDataSource;
selectorAddress.mScope = kAudioDevicePropertyScopeInput;
selectorAddress.mElement = kAudioObjectPropertyElementMain;
// Request microphone permission
status = AudioObjectSetPropertyData(deviceID, &selectorAddress, 0, nullptr, sizeof(CFStringRef), "Microphone");
if(status != noErr) {
printf("Error: %d\n", status);
}
printf("Request microphone permission: %d\n", status);
}
int main()
{
AudioDeviceID deviceID;
UInt32 size = sizeof(deviceID);
AudioObjectPropertyAddress propertyAddress = {
kAudioHardwarePropertyDefaultInputDevice,
kAudioObjectPropertyScopeGlobal,
kAudioObjectPropertyElementMain
};
AudioObjectGetPropertyData(kAudioObjectSystemObject, &propertyAddress, 0, NULL, &size, &deviceID);
requestMicrophonePermission(deviceID);
printf("Device ID: %u\n", deviceID);
AudioObjectPropertyAddress propertyAddress2 = {
kAudioDevicePropertyStreamConfiguration,
kAudioObjectPropertyScopeInput,
kAudioObjectPropertyElementMain
};
OSStatus status;
UInt32* size2;
AudioBufferList bufferList[] = {3};
status = AudioObjectGetPropertyData(
deviceID, &propertyAddress2, 0, NULL, size2, bufferList);
printf("Audio buffer: %u\n", bufferList[0].mNumberBuffers);
printf("Audio data: %u\n", bufferList[0].mBuffers[0].mNumberChannels);
printf("Audio data: %u\n", bufferList->mBuffers[0].mNumberChannels);
// getAudioBufferListFromDeviceID(deviceID);
return 0;
}The Apple Silicon Mac returns different errors depending on the input source.
- Scarlett 8i6 USB
Error: 2003332927 - Macbook pro microphone
Error: 1852797029 - HD Pro Webcam C920
Error: 2003332927
Running the same code on the Intel mac returns 0 so no error on the status
I'm talking about this part of the code
// Request microphone permission
status = AudioObjectSetPropertyData(deviceID, &selectorAddress, 0, nullptr, sizeof(CFStringRef), "Microphone");
if(status != noErr) {
printf("Error: %d\n", status);
}
printf("Request microphone permission: %d\n", status);So we need to request permission for using the sound card from a CLI program? How very interesting, and annoying. Thank you for the analysis.
Is this why we can't get the channel number?
@bastibe Any plans to support MacOS? I believe @alfonsocv12 has given some possible way out for the problems.
I don't currently have the time to do it myself. But I'll happily review a pull request.
I'm trying to make some headway into this issue, but so far no success.
It seems to me that all the relevant APIs mentioned in the documentation are only available to Objective-C, not C.
Please, please, someone tell me I'm wrong. If not, soundcard is essentially dead on macOS.
Let me give it some time, I can help.
Currently I have my project working with c++ and a library called RTAudio.
On both Apple Silicon and Intel based mac's
I playing with this a bit on the terminal, I got a permission prompt automatically. So it doesn't look like you need to request it manually.
However, that doesn't solve the channel issue at all
(additionally, I get a block size error later on)
I think I found the error actually.
Comparing things between Stackoverflow posts (Swift/Objc Code) and this project, I noticed a few things.
- _CoreAudio.get_property is wrongly implemented (I think)
"AudioBufferList" is in itself a struct with lists, so returning it as an array is wrong
So, I did a quick & dirty implementation like this:
def get_property2(target, selector, ctype, scope=_cac.kAudioObjectPropertyScopeGlobal):
"""Get a CoreAudio property.
This might include things like a list of available sound
cards, or various meta data about those sound cards.
Arguments:
- `target`: The AudioObject that the property belongs to
- `selector`: The Selector for this property
- `scope`: The Scope for this property
- `ctype`: The type of the property
Returns:
A list of objects of type `ctype`
"""
prop = _ffi.new("AudioObjectPropertyAddress*",
{'mSelector': selector,
'mScope': scope,
'mElement': _cac.kAudioObjectPropertyElementMaster})
has_prop = _ca.AudioObjectHasProperty(target, prop)
assert has_prop == 1, 'Core Audio does not have the requested property'
size = _ffi.new("UInt32*")
err = _ca.AudioObjectGetPropertyDataSize(target, prop, 0, _ffi.NULL, size)
assert err == 0, "Can't get Core Audio property size"
prop_data = _ffi.new(ctype + '*')
err = _ca.AudioObjectGetPropertyData(target, prop, 0, _ffi.NULL,
size, prop_data)
assert err == 0, "Can't get Core Audio property data"
return prop_data
- There is a small difference in the headers for
AudioBufferListincoreaudio.py.h
Apple's file is defined as (source: https://github.com/phracker/MacOSX-SDKs/blob/041600eda65c6a668f66cb7d56b7d1da3e8bcc93/MacOSX11.3.sdk/System/Library/Frameworks/CoreAudioTypes.framework/Versions/A/Headers/CoreAudioBaseTypes.h#L160)
AudioBuffer mBuffers[1]; // this is a variable length array of mNumberBuffers elements
vs ours
AudioBuffer mBuffers[]; // this is a variable length array of mNumberBuffers elements
Sounds silly but with the above I can do this:
def channels(self):
bufferlist = _CoreAudio.get_property2(
self._id,
_cac.kAudioDevicePropertyStreamConfiguration,
'AudioBufferList', scope=_cac.kAudioObjectPropertyScopeInput)
numChannels = 0
if bufferlist and bufferlist.mNumberBuffers > 0:
for buffer in bufferlist.mBuffers:
numChannels += buffer.mNumberChannels
return numChannels
And it will return
[<Microphone Londo Mollari Microphone (1 channels)>, <Microphone Patrice CGM Microphone (1 channels)>, <Microphone BlackHole 2ch (2 channels)>, <Microphone MacBook Pro Microphone (1 channels)>, <Microphone MacBook Pro Speakers (0 channels)>, <Microphone Camo Microphone (2 channels)>, <Microphone Microsoft Teams Audio (2 channels)>, <Microphone Elmedia OUT (2 channels)>, <Microphone Stream OUT (2 channels)>, <Microphone Zoom OUT (2 channels)>, <Microphone ZoomAudioDevice (2 channels)>, <Microphone Multi-Output Device (0 channels)>]```
That is terrific news! Several people (including myself) have tried to solve this issue in the past, and finally, you managed to solve it! Fantastic job, and thank you so, so much!
Would you like to contribute your fix as a pull request?
@bastibe Small problem with that unfortunately. I don't know enough about this project to know how to implement it properly. I can hack it to work for me, but I may break other functionality
@drallgood THANK YOU SO MUCH!!!!
You finally, finally, discovered a really stupid bug that has been lurking in SoundCard/macOS for many years:
The code used to
prop_data = _ffi.new(ctype+'[]', num_values)
err = _ca.AudioObjectGetPropertyData(target, prop, 0, _ffi.NULL,
size, prop_data)
assert err == 0, "Can't get Core Audio property data"
return [prop_data[idx] for idx in range(num_values)]
This seems innocuous enough if prop_data[idx] contains simple data such as an int. However, if prop_data[idx] is, say, a struct, then that struct is still owned by prop_data, and will get freed once we leave the function. This is the case when we query the number of channels in a soundcard: We return a pointer to an AudioBufferList, which is immediately collected and deallocated on the next garbage collection run. On deallocation, the mNumberBuffers is zero'd out, and the number of channels is therefore interpreted as empty, and the soundcard is discarded.
To fix this, we need to
return prop_data
which passes ownership to the receiving function, and everything works as intended.
Such a simple change, but such dire consequences.
I bet this worked in the past due to some garbage collection change. If we're fast enough, and access the AudioBufferList before the garbage collector collects prop_data, we can read the number of channels before they are deallocated. But this has always been a bug, and should never have worked.
I have spent many an hour on this problem in the last few years, but it has eluded me. So once again, THANK YOU SO MUCH!
Originally, I was going to ask you to submit the fix in your name, but there were some follow-up errors to fix, and it became too complicated.
Still, I'd be grateful if you could review/try out the fix in #177, and report back if the fix works on your end!
I just uploaded version 0.4.3, which fixes the issue.