dlopen fails to find Vulkan libraries on macOS
kosude opened this issue · 6 comments
When using volk on macOS (Apple M1), volkInitialise() fails with INITIALIZATION_FAILED due to not finding libvulkan or libMoltenVK. These files are present on my system, in the /usr/local/lib/ directory, yet here, module remains NULL.
This seems to only occur on macOS.
After some testing, I found that, as far as I can tell, the first commit where this error is present is 3f3e0cb.
I don't think the fact that this error is absent before the commit is indicative of anything as I suspect the reason the commit right before that one works is that in your CMake configuration prior to that commit, libvulkan.dylib ended up being linked into the application binary, so dlopen found an already existing module.
I'd need to reproduce and debug this. How did you install MoltenVK?
Regarding my CMake configuration; I've tested this with no changes to my build system other than the volk version. (Unless I misunderstood - I interpreted that as referring to my CMakeLists.txt files, which have been kept the same). Upon reviewing my build setup, I'm not aware of any other places where libvulkan.dylib could have been linked.
If I remember correctly, I installed MoltenVK as part of the SDK.
What I mean essentially is that if volk worked before the commit you referenced, it did for the wrong reason as the dynamic library discovery didn’t actually run. I’ll try this with the SDK install.
It looks like /usr/local/lib is not used as a search path by default when using dlopen. Running the test without any extra setup I can reproduce this error, but it is fixed by running DYLD_LIBRARY_PATH=/usr/local/lib ./volk_test. This can also be fixed by running DYLD_FALLBACK_LIBRARY_PATH=/usr/local/lib ./volk_test. Both variables don't have a value on my system; this is confusing because it conflicts the documentation here: https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/dlopen.3.html
Hmm, man dlopen says:
Lastly, for old binaries, dyld will try some fallbacks. If $DYLD_FALLBACK_LIBRARY_PATH was set at launch, dyld will search in those directories, otherwise, dyld will look in /usr/local/lib/ (if the
process is unrestricted), and then in /usr/lib/.
It doesn't say what exactly "old binaries" mean; this behavior (ignoring /usr/local/lib by default) seems to also hold for x86_64 binaries compiled while targeting older OS versions (e.g. compiling using clang -arch x86_64 -mmacosx-version-min=10.9).
Using DYLD_PRINT_APIS=1 I can confirm that dyld doesn't even look in usr/local/lib, but does look in other folders. Maybe this is a silent behavior change from some macOS version.
dyld[3305]: dlopen(libvulkan.dylib) => NULL, 'dlopen(libvulkan.dylib, 0x0006): tried: 'libvulkan.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibvulkan.dylib' (no such file), '/usr/lib/libvulkan.dylib' (no such file, not in dyld cache), 'libvulkan.dylib' (no such file)'
I suspect the only reasonable option for now is to add one more fallback path and try /usr/local/lib/libvulkan.dylib if all other paths fail to load. This is a little awkward as normally I'd expect the external configuration to handle this, but it's better than not working...