Crash on macOS Big Sur 11.4
Closed this issue · 19 comments
I get the following error when using darkdetect from the terminal:
>>> darkdetect.isDark()
2021-06-08 19:36:55.831 Python[4299:381204] +[NSUserDefaults standardUserDefaults]: unrecognized selector sent to class 0x7fff8011ea10
2021-06-08 19:36:55.835 Python[4299:381204] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '+[NSUserDefaults standardUserDefaults]: unrecognized selector sent to class 0x7fff8011ea10'
*** First throw call stack:
(
0 CoreFoundation 0x00007fff2055287b __exceptionPreprocess + 242
1 libobjc.A.dylib 0x00007fff2028ad92 objc_exception_throw + 48
2 CoreFoundation 0x00007fff205d52e5 __CFExceptionProem + 0
3 CoreFoundation 0x00007fff204ba90b ___forwarding___ + 1448
4 CoreFoundation 0x00007fff204ba2d8 _CF_forwarding_prep_0 + 120
5 libffi.dylib 0x00007fff2d9ba8f5 ffi_call_unix64 + 85
6 ??? 0x00007ffeee3d67e0 0x0 + 140732895422432
)
libc++abi: terminating with uncaught exception of type NSException
zsh: abort python3
This happens when using any method, i.e., isDark
, isLight
, or theme
. For information, I'm running macOS Big Sur 11.4 (20F71).
Also on Big Sur:
Python 3.9.5 (default, Aug 20 2021, 17:29:18)
[Clang 12.0.5 (clang-1205.0.22.11)] on darwin
Type "help", "copyright", "credits" or "license" for more information.import darkdetect
print(darkdetect.theme())
[1] 87474 segmentation fault python
Also on Big Sur:
Python 3.9.5 (default, Aug 20 2021, 17:29:18)
[Clang 12.0.5 (clang-1205.0.22.11)] on darwin
Type "help", "copyright", "credits" or "license" for more information.import darkdetect
print(darkdetect.theme())
[1] 87474 segmentation fault python
Are you using an M1 Mac or Intel Mac?
Also could you trying running this code, for a bit more debugging info?
import faulthandler
faulthandler.enable()
import darkdetect
print(darkdetect.theme())
I'm on Big Sur 11.5.2, but M1 MBP and having this same issue.
I used the faulthandler as instructed above in ipython:
Python 3.9.6 | packaged by conda-forge | (default, Jul 6 2021, 08:51:19)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.25.0 -- An enhanced Interactive Python. Type '?' for help.
Fatal Python error: Segmentation fault
Thread 0x00000001705db000 (most recent call first):
File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/threading.py", line 312 in wait
File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/threading.py", line 574 in wait
File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/core/history.py", line 829 in run
File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/core/history.py", line 58 in needs_sqlite
File "<decorator-gen-17>", line 2 in run
File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/threading.py", line 973 in _bootstrap_inner
File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/threading.py", line 930 in _bootstrap
Current thread 0x0000000100f43d40 (most recent call first):
File "/Users/piotrsobolewski/Dev/napari/napari/_vendor/darkdetect/_mac_detect.py", line 44 in theme
File "<ipython-input-3-30534ba1feb8>", line 1 in <module>
File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3441 in run_code
File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3361 in run_ast_nodes
File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3169 in run_cell_async
File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/core/async_helpers.py", line 68 in _pseudo_sync_runner
File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 2944 in _run_cell
File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 2898 in run_cell
File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/terminal/interactiveshell.py", line 555 in interact
File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/terminal/interactiveshell.py", line 564 in mainloop
File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/terminal/ipapp.py", line 356 in start
File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/traitlets/config/application.py", line 845 in launch_instance
File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/__init__.py", line 126 in start_ipython
File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/bin/ipython", line 8 in <module>
fish: Job 1, 'ipython' terminated by signal SIGSEGV (Address boundary error)
Wonder if this is related to the fact that running:
defaults read -globalDomain AppleInterfaceStyle
from the Terminal on Big Sur returns an error while in Light mode:
2021-09-11 19:39:43.262 defaults[63934:11811421]
The domain/default pair of (kCFPreferencesAnyApplication, AppleInterfaceStyle) does not exist
The same command, when run in Dark mode returns:
Dark
Hmmm... interesting, perhaps using that command would be more reliable than using ctypes
. Perhaps something like:
import subprocess
def theme():
sub = subprocess.Popen("defaults read -globalDomain AppleInterfaceStyle".split(), stdout=subprocess.PIPE)
result = sub.stdout.read()
if isinstance(result, bytes):
try:
result = result.decode()
except UnicodeDecodeError:
return 'Light'
if result[-1] == '\n':
result = result[:-1]
return 'Dark' if result == 'Dark' else 'Light'
Not sure how this works with other versions of macOS though. Works well on my Intel Mac running Big Sur 11.4.
I've used this python hack of a terminal command:
import os
test = os.popen('defaults read -globalDomain AppleInterfaceStyle &> /dev/null && echo dark || echo light')
theme = test.read().strip()
Works on Big Sur M1.
Edit: Can report your code above using subprocess
works in Dark mode, but not light:
In [3]: theme()
Out[3]: 'Dark'
<<<switch to light>>>
In [4]: theme()
2021-09-11 22:52:05.102 defaults[65849:11909460]
The domain/default pair of (kCFPreferencesAnyApplication, AppleInterfaceStyle) does not exist
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-5-8bbb2e4d564a> in <module>
----> 1 theme()
<ipython-input-2-b9ad10b0edfb> in theme()
7 except UnicodeDecodeError:
8 return 'Light'
----> 9 if result[-1] == '\n':
10 result = result[:-1]
11 return 'Dark' if result == 'Dark' else 'Light'
IndexError: string index out of range
Thanks to all for reporting and investigating this. I have also experienced this issue on Big Sur 11.3+ and an Intel Mac.
What I find intriguing is that the issue occurs only when running the code from Terminal. If the code is called by a proper .app bundle (e.g. one frozen with py2app), then the detection always works as expected.
To me, this and the error message we see ([NSUserDefaults standardUserDefaults]: unrecognized selector sent to class
) point to some context missing in the Terminal app. Perhaps we need to create this context manually In darkdetect
to make sure the detection works also from Terminal. Alas, I have no idea how. Perhaps some Cocoa developer can jump in.
@psobolewskiPhD @Ahsoka Detection methods based on command invocations + subprocess
usually fails when executed from a frozen application (e.g. a py2app
bundle). I coded darkdetect
with ctypes
precisely to be able to detect the Dark mode from a frozen Python application.
Maybe when not frozen you can use a command invocation for detection and when frozen use the ctypes
approach.
Psuedo code:
import sys
def theme():
if getattr(sys, 'frozen', False):
ctypes_method()
else:
command_invocation_method()
I found the issue: libobjc.dylib
was not correctly loaded by ctypes
due to a change in Big Sur:
New in macOS Big Sur 11 beta, the system ships with a built-in dynamic linker cache of all system-provided libraries. As part of this change, copies of dynamic libraries are no longer present on the filesystem. Code that attempts to check for dynamic library presence by looking for a file at a path or enumerating a directory will fail. Instead, check for library presence by attempting to dlopen () the path, which will correctly check for the library in the cache. (62986286)
I now wonder how this worked from .app bundles, perhaps ctypes.util.find_library
is able to locate library files when called from there.
Patching
darkdetect/darkdetect/_mac_detect.py
Lines 10 to 11 in aa8a78e
appkit = ctypes.cdll.LoadLibrary('AppKit.framework/AppKit')
objc = ctypes.cdll.LoadLibrary('libobjc.dylib')
allows me to correctly run the package from Terminal on Big Sur 11.3.1.
I assume the change is not compatible with older macOS versions so, a proper fix will require to retain the old code for backward compatibility. Also, the effects of this change to .app bundles needs to be investigated. I intend to do this as soon as I have some spare time, then release a new version of darkdetect
containing this patch.
Thanks again to all the people that contributed to this discussion.
(Credits for the help I found in vispy/vispy#1885 and vispy/vispy#1975).
Fixed in 0.4.0.
Alas, the issue is not resolved for me Big Sur 11.5.2 but on M1 mac
Collecting darkdetect
Downloading darkdetect-0.4.0-py3-none-any.whl (6.3 kB)
Installing collected packages: darkdetect
Successfully installed darkdetect-0.4.0
╭─ ~ ···································································· ✔ ─╮
╰─ ipython (napari-test) ─╯
Python 3.9.6 | packaged by conda-forge | (default, Jul 6 2021, 08:51:19)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.25.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import darkdetect
In [2]: darkdetect.theme()
fish: Job 1, 'ipython' terminated by signal SIGSEGV (Address boundary error)
Edit: I tested in both light and dark mode.
Hmmm... very interesting, the issue is resolved for me on my Intel Mac and it appears that @albertosottile also has an Intel Mac. Seems like a M1 Mac specific issue.
That's what I feared 🦑
I thought it was related to 3.9.6 vs 3.9.7, so I updated python, but same:
╰─ ipython (napari-test) ─╯
Python 3.9.7 | packaged by conda-forge | (default, Sep 2 2021, 17:55:16)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.25.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import darkdetect
In [2]: darkdetect.theme()
fish: Job 1, 'ipython' terminated by signal SIGSEGV (Address boundary error)
@psobolewskiPhD It appears we have two different issues at hand. Unfortunately, I do not have a M1 Mac to reproduce/debug this one. However, in this case I would ask you to open another issue on GitHub to better distinguish the latter bug from the one that I just patched.
In case you want to do that, it might be a good idea to remove conda and ipython from the equation, and run directly darkdetect
from the system Python interpreter (perhaps after importing faulthandler
as suggested by @Ahsoka ). If you do not want to install the package, you can run the test script by just checking out the git repository, as this package has no requirements.
I think I've narrowed it down.
I think it's related to the objc_msgSend
, based on this:
https://developer.apple.com/documentation/apple-silicon/addressing-architectural-differences-in-your-macos-code
At the top we have:
objc.objc_msgSend.argtypes = [void_p, void_p]
But in def theme():
appearanceNS = msg(stdUserDef, n('stringForKey:'), void_p(key))
This triggers segfault.
You can do a simple test by using:
NSString = C('NSString')
objc.objc_msgSend(NSString, n("stringWithUTF8String:"), _utf8("reason"))
If the .argtypes is just [void_p, void_p] it segfaults, if it's [void_p, void_p, void_p] it runs.
I've fixed it locally, by making the .argtypes have 3 arguments and then passing None as the third everywhere. Not very elegant, but it works. I can open issue/PR if you think that's a way forward.
BTW, not sure why but at least on my system (now 3.9.7) the new V check code isn't needed.
This works just fine:
appkit = ctypes.cdll.LoadLibrary(ctypes.util.find_library('AppKit'))
objc = ctypes.cdll.LoadLibrary(ctypes.util.find_library('objc'))
Edit: just to show:
appkit = ctypes.cdll.LoadLibrary(ctypes.util.find_library('AppKit'))
output:
<CDLL '/System/Library/Frameworks/AppKit.framework/AppKit', handle 115edc6f0 at 0x1075a6850>
@psobolewskiPhD I'd definitely open a new issue and report/resume this discussion there.
@psobolewskiPhD Re: the need of the "V" patch. I suspect eventually the Python teams would integrate/had integrated the patch within ctypes
itself, which is probably what happened in 3.9.7. Nevertheless, I would like to retain compatibility with older minor versions, hence the need to add this patch.