Maybe need add `libwebp.7.dylib` to `Frameworks` folder?
Closed this issue · 24 comments
Problems:
After build the emacs, and codesign (#64). My mac return me the panic report that:
dyld[16543]: Library not loaded: @rpath/libwebp.7.dylib
Referenced from: <8B34F8AD-74B5-371E-AED8-5F8BC7340372> /Applications/Emacs.app/Contents/Frameworks/libwebpdemux.2.dylib
Reason: tried: '/Applications/Emacs.app/Contents/Frameworks/../lib/libwebp.7.dylib' (no such file), '/Applications/Emacs.app/Contents/Frameworks/libwebp.7.dylib' (no such file), '/Applications/Emacs.app/Contents/Frameworks/libwebp.7.dylib' (no such file), '/usr/local/lib/libwebp.7.dylib' (no such file), '/usr/lib/libwebp.7.dylib' (no such file, not in dyld cache)
Temporary solution:
After some research and debugging, I find it the @rpath value is path @executable_path/../Frameworks (offset 12). After I add homebrew's lib path to @rpath by install_name_tool -add_rpath /opt/homebrew/lib /Applications/Emacs.app/Contents/MacOS/Emacs. The problem is solved. It can get the libwebp.7.dylib now.
I checked the code and find line 170 define extra_libs which copy to Frameworks in embed (maybe? I don't know Ruby).
Maybe we can also copy libwebp.7.dylib too? Thanks!
My env:
macOS 13.4.1
M2 Max
GNU Emacs master branch
After manually copy libwebp.7.dylib to Framework, I find there is another thing need to copy to Framework. libsharpyuv.0.dylib also need to copy to Framework
cp /opt/homebrew/lib/libsharpyuv.0.dylib /Applications/Emacs.app/Contents/Frameworks/
Maybe script can add an optional config that add the homebrew lib path to @rpath if we don't want to embedding?
I ran into the same issues building 29.0.2, and #86 fixes the issue for me. So 👍 from me fwiw. ;)
Thanks for the report and PR @ccqpein. I’ll try and find some time to have a proper look at this before the end of the week.
I have a feeling it’s a combination of new optional configure flags in emacs which automatically enable themselves if specific brew packages happen to be installed, along with what looks like a potential bug with the shared library embedding code in the build script.
@ccqpein Apologies, it took me longer to get around to this than I thought. However I can't re-produce it on my own machines. I don't have any Apple Silicon machines available to me, but I don't think it's an Intel vs AS issue.
For what it's worth, in my pre-builds project, the 29.0.92-pretest build for example includes the libwebp7.dylib and libsharpyuv.0.dylib files just fine.
So I'm not fully sure what might be going on. The fact that it's linked to via @rpath, means the lib embedder logic detects it as being a shared library from within Homebrew, and hence copies it into the app bundle and updates the share library link. Possibly there was some random glitch while performing the file copy operation.
Since I can't re-produce it myself, I'm afraid I might need to ask you debug things a bit further. I have just pushed a new v0.6.42 of build-emacs-for-macos, which has a new --log-level flag. If you set the log level to debug, you'll get a lot more details printed towards the end when the various libraries embedding/bundling logic runs.
You can also try to use the --no-archive option to skip the archiving step at the end, leaving you with a folder under builds to speed things up. Then you can compare sources/**/nextstep/Emacs.app and builds/**/Emacs.app, the later of which will have had all the library embedding steps performed. Between the debug output and otool -L results for both apps, hopefully something might become obvious.
Let me know if there's anything more information I can give you, or help you with further debugging in general.
@jimeh no need to apologies. I am trying to rebuild with your update now. I will keep this issue updated. Thank you!
There are outputs of build-emacs-for-macos/builds/Emacs.2023-08-01.74d6604.master.macOS-13.arm64/Emacs.app/Contents/MacOS/Emacs and /build-emacs-for-macos/sources/emacs-mirror-emacs-74d6604/nextstep/Emacs.app/Contents/MacOS/Emacs with the --no-archive argument:
/build-emacs-for-macos/builds/Emacs.2023-08-01.74d6604.master.macOS-13.arm64/Emacs.app/Contents/MacOS/Emacs:
/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit (compatibility version 45.0.0, current version 2299.50.120)
/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
/System/Library/Frameworks/Carbon.framework/Versions/A/Carbon (compatibility version 2.0.0, current version 169.0.0)
/System/Library/Frameworks/IOSurface.framework/Versions/A/IOSurface (compatibility version 1.0.0, current version 1.0.0)
/System/Library/Frameworks/QuartzCore.framework/Versions/A/QuartzCore (compatibility version 1.2.0, current version 1.11.0)
/System/Library/Frameworks/UniformTypeIdentifiers.framework/Versions/A/UniformTypeIdentifiers (compatibility version 1.0.0, current version 709.0.0)
@rpath/libtiff.6.dylib (compatibility version 7.0.0, current version 7.1.0)
@rpath/libjpeg.8.dylib (compatibility version 8.0.0, current version 8.3.2)
@rpath/libpng16.16.dylib (compatibility version 57.0.0, current version 57.0.0)
@rpath/libgif.dylib (compatibility version 0.0.0, current version 7.2.0)
@rpath/libwebpdemux.2.dylib (compatibility version 3.0.0, current version 3.13.0)
@rpath/libwebpdecoder.3.dylib (compatibility version 5.0.0, current version 5.7.0)
/System/Library/Frameworks/WebKit.framework/Versions/A/WebKit (compatibility version 1.0.0, current version 615.1.26)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
@rpath/libgnutls.30.dylib (compatibility version 66.0.0, current version 66.0.0)
@rpath/liblcms2.2.dylib (compatibility version 3.0.0, current version 3.15.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
@rpath/libjansson.4.dylib (compatibility version 19.0.0, current version 19.0.0)
@rpath/libgmp.10.dylib (compatibility version 15.0.0, current version 15.1.0)
@rpath/libgccjit.0.dylib (compatibility version 0.0.0, current version 0.0.1)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.100.3)
@rpath/libtree-sitter.0.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libsqlite3.dylib (compatibility version 9.0.0, current version 346.0.0)
/System/Library/Frameworks/CFNetwork.framework/Versions/A/CFNetwork (compatibility version 1.0.0, current version 1406.0.4)
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1971.0.0)
/System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics (compatibility version 64.0.0, current version 1690.5.4)
/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 1228.0.0)
/System/Library/Frameworks/CoreText.framework/Versions/A/CoreText (compatibility version 1.0.0, current version 1.0.0)
/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1971.0.0)
/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
/build-emacs-for-macos/sources/emacs-mirror-emacs-74d6604/nextstep/Emacs.app/Contents/MacOS/Emacs:
/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit (compatibility version 45.0.0, current version 2299.50.120)
/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
/System/Library/Frameworks/Carbon.framework/Versions/A/Carbon (compatibility version 2.0.0, current version 169.0.0)
/System/Library/Frameworks/IOSurface.framework/Versions/A/IOSurface (compatibility version 1.0.0, current version 1.0.0)
/System/Library/Frameworks/QuartzCore.framework/Versions/A/QuartzCore (compatibility version 1.2.0, current version 1.11.0)
/System/Library/Frameworks/UniformTypeIdentifiers.framework/Versions/A/UniformTypeIdentifiers (compatibility version 1.0.0, current version 709.0.0)
/opt/homebrew/opt/libtiff/lib/libtiff.6.dylib (compatibility version 7.0.0, current version 7.1.0)
/opt/homebrew/opt/jpeg-turbo/lib/libjpeg.8.dylib (compatibility version 8.0.0, current version 8.3.2)
/opt/homebrew/opt/libpng/lib/libpng16.16.dylib (compatibility version 57.0.0, current version 57.0.0)
/opt/homebrew/opt/giflib/lib/libgif.dylib (compatibility version 0.0.0, current version 7.2.0)
/opt/homebrew/opt/webp/lib/libwebpdemux.2.dylib (compatibility version 3.0.0, current version 3.13.0)
/opt/homebrew/opt/webp/lib/libwebpdecoder.3.dylib (compatibility version 5.0.0, current version 5.7.0)
/System/Library/Frameworks/WebKit.framework/Versions/A/WebKit (compatibility version 1.0.0, current version 615.1.26)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/opt/homebrew/opt/gnutls/lib/libgnutls.30.dylib (compatibility version 66.0.0, current version 66.0.0)
/opt/homebrew/opt/little-cms2/lib/liblcms2.2.dylib (compatibility version 3.0.0, current version 3.15.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/opt/homebrew/opt/jansson/lib/libjansson.4.dylib (compatibility version 19.0.0, current version 19.0.0)
/opt/homebrew/opt/gmp/lib/libgmp.10.dylib (compatibility version 15.0.0, current version 15.1.0)
/opt/homebrew/opt/libgccjit/lib/gcc/current/libgccjit.0.dylib (compatibility version 0.0.0, current version 0.0.1)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.100.3)
/opt/homebrew/opt/tree-sitter/lib/libtree-sitter.0.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libsqlite3.dylib (compatibility version 9.0.0, current version 346.0.0)
/System/Library/Frameworks/CFNetwork.framework/Versions/A/CFNetwork (compatibility version 1.0.0, current version 1406.0.4)
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1971.0.0)
/System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics (compatibility version 64.0.0, current version 1690.5.4)
/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 1228.0.0)
/System/Library/Frameworks/CoreText.framework/Versions/A/CoreText (compatibility version 1.0.0, current version 1.0.0)
/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1971.0.0)
/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
Then I searched the output of buildin. For libwebp, there is some log:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: warning: changes being made to the file will invalidate the code signature in: /Users/{user}/build-emacs-for-macos/builds/Emacs.2023-08-01.74d6604.master.macOS-13.arm64/Emacs.app/Contents/MacOS/Emacs
==> DEBUG: -- Bundling shared library: /opt/homebrew/opt/webp/lib/libwebpdemux.2.dylib
==> DEBUG: -- -- Copying to: Contents/Frameworks/libwebpdemux.2.dylib
==> DEBUG: -- -- Relinking to: @rpath/libwebpdemux.2.dylib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: warning: changes being made to the file will invalidate the code signature in: /Users/{user}/build-emacs-for-macos/builds/Emacs.2023-08-01.74d6604.master.macOS-13.arm64/Emacs.app/Contents/MacOS/Emacs
==> DEBUG: -- Bundling shared library: /opt/homebrew/opt/webp/lib/libwebpdecoder.3.dylib
==> DEBUG: -- -- Copying to: Contents/Frameworks/libwebpdecoder.3.dylib
==> DEBUG: -- -- Relinking to: @rpath/libwebpdecoder.3.dylib
But I don't see anything about Looks like both missing lib files are came from libwebplibsharpyuv.
@ccqpein I believe I've tracked down the root of the issue. The original libwebpdemux.2.dylib file links to other webp shared libraries by using @rpath, which the library embedder does not understand at the moment. I'll try and get a fix in place soon.
I just build the Emacs master branch, and looks like there are some dylib missing too. But this time from @load_path
dyld[55871]: Library not loaded: @loader_path/../../../../opt/gettext/lib/libintl.8.dylib
Referenced from: <A95699CE-2BA7-3EA0-A074-A47CAB443C4D> /Applications/Emacs.app/Contents/Frameworks/libgnutls.30.dylib
Reason: tried: '/Applications/Emacs.app/Contents/Frameworks/../../../../opt/gettext/lib/libintl.8.dylib' (no such file), '/usr/local/lib/libintl.8.dylib' (no such file), '/usr/lib/libintl.8.dylib' (no such file, not in dyld cache)
Looks like it still the old load_path. The one in M2 chip mac should be
/Applications/Emacs.app/Contents/Frameworks/../../../../opt/homebrew/opt/gettext/lib/libintl.8.dylib
instead of
/Applications/Emacs.app/Contents/Frameworks/../../../../opt/gettext/lib/libintl.8.dylib
Thanks for the details :)... Thankfully I've already come across that one too. It's because the shared library links to other shared libraries using @load_path, which confuses the custom shared library embedding/bundling code in the build script.
These issues have necessitated a rewrite of said code, and I have a nearly working version of it locally. I just need to find some more time this week to iron out a few bugs before it'll work properly.
Hi @jimeh. Any news on this? :)
I have the following which prevents me from starting Emacs:
dyld[43042]: Library not loaded: @loader_path/../../../../opt/freetype/lib/libfreetype.6.dylib
Referenced from: <C9101C0F-3D61-3F2C-8DBD-F79F68FD092A> /Applications/Emacs.app/Contents/Frameworks/libharfbuzz.0.dylib
Reason: tried: '/Applications/Emacs.app/Contents/Frameworks/../../../../opt/freetype/lib/libfreetype.6.dylib' (no such file), '/usr/local/lib/libfreetype.6.dylib' (no such file), '/usr/lib/libfreetype.6.dylib' (no such file, not in dyld cache)Library not loaded: @loader_path/../../../../opt/icu4c/lib/libicui18n.73.dylib
Referenced from: <FF684384-280D-3F59-8741-D9AE10E07338> /Applications/Emacs.app/Contents/Frameworks/libxml2.2.dylib
Reason: tried: '/Applications/Emacs.app/Contents/Frameworks/../../../../opt/icu4c/lib/libicui18n.73.dylib' (no such file), '/usr/local/lib/libicui18n.73.dylib' (no such file), '/usr/lib/libicui18n.73.dylib' (no such file, not in dyld cache)Library not loaded: @loader_path/../../../../opt/gettext/lib/libintl.8.dylib
Referenced from: <A95699CE-2BA7-3EA0-A074-A47CAB443C4D> /Applications/Emacs.app/Contents/Frameworks/libgnutls.30.dylib
Reason: tried: '/Applications/Emacs.app/Contents/Frameworks/../../../../opt/gettext/lib/libintl.8.dylib' (no such file), '/usr/local/lib/libintl.8.dylib' (no such file), '/usr/lib/libintl.8.dylib' (no such file, not in dyld cache)
Thanks!
Hi @jimeh How is everything going? Do you need more test or log info I can help with? Thanks.
I'm also experiencing this on my M1 Macbook Pro, running Ventura 13.5.2.
But the fix in #86 is not enough for me; with those two files copied into Frameworks I get this:
dyld[92706]: Library not loaded: @loader_path/../../../../opt/icu4c/lib/libicui18n.73.dylib
Referenced from: <FF684384-280D-3F59-8741-D9AE10E07338> /Applications/Emacs.app/Contents/Frameworks/libxml2.2.dylib
Reason: tried: '/Applications/Emacs.app/Contents/Frameworks/../../../../opt/icu4c/lib/libicui18n.73.dylib' (no such file), '/usr/local/lib/libicui18n.73.dylib' (no such file), '/usr/lib/libicui18n.73.dylib' (no such file, not in dyld cache)
Library not loaded: @loader_path/../../../../opt/gettext/lib/libintl.8.dylib
Referenced from: <A95699CE-2BA7-3EA0-A074-A47CAB443C4D> /Applications/Emacs.app/Contents/Frameworks/libgnutls.30.dylib
Reason: tried: '/Applications/Emacs.app/Contents/Frameworks/../../../../opt/gettext/lib/libintl.8.dylib' (no such file), '/usr/local/lib/libintl.8.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/usr/lib/libintl.8.dylib' (no such file, not in dyld cache)
and indeed libxml2.2.dylib looks for libicui18n.73.dylib in this path:
% otool -L /Applications/Emacs.app/Contents/Frameworks/libxml2.2.dylib
/Applications/Emacs.app/Contents/Frameworks/libxml2.2.dylib:
@rpath/libxml2.2.dylib (compatibility version 14.0.0, current version 14.5.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
@loader_path/../../../../opt/icu4c/lib/libicui18n.73.dylib (compatibility version 73.0.0, current version 73.2.0)
@loader_path/../../../../opt/icu4c/lib/libicuuc.73.dylib (compatibility version 73.0.0, current version 73.2.0)
@loader_path/../../../../opt/icu4c/lib/libicudata.73.dylib (compatibility version 73.0.0, current version 73.2.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.100.3)
My build log (just the bundling part) is in this gist: https://gist.github.com/garyo/6a6dec778dfa98c35e4a03623df21ce6 in case it's helpful.
Apologies for the radio silence everyone. Tail end of the summer got a bit crazy so I've not had any time available to look at this (and a few other things). Things are returning to normal this month though, so I'll be getting this sorted out soon.
FWIW, I did these additional commands and now it's working on my M1 mac:
for libname in libicui18n.73.dylib libicuuc.73.dylib libicudata.73.dylib ; do
cp /opt/homebrew/Cellar/icu4c/73.2/lib/$libname builds/Emacs.2023-10-11.cfed3bb.master.macOS-13.arm64/Emacs.app/Contents/Framework;
install_name_tool -change @loader_path/../../../../opt/icu4c/lib/$libname @rpath/$libname builds/Emacs.2023-10-11.cfed3bb.master.macOS-13.arm64/Emacs.app/Contents/Frameworks/libxml2.2.dylib;
echo $libname done;
done
@jimeh Hope everything is fine. Let me know if you need more logs or test. Thanks!
Is there any workaround for this issue? This appears to be the only Emacs build that offers casks with native-comp, I would love to help out here in any way I can.
Just a quick note to say that @jimeh 's refactor-shared-library-bundler-logic branch is actually 100% working for me. I know the git log says it's not working yet and I'm sure there is work to be done still but a basic no-args build works fine on my M1 Macbook running Ventura. (I had to install a newer ruby via rbenv to avoid some build problems, but then the build worked fine.)
@garyo @ccqpein That's great to hear, and thanks for testing that branch out. Last time I touched it, it was failing spectacularly for me. But it was middle of the night and I was half asleep, so I was probably doing something stupid... lol
I've got time set aside over the next couple of weeks to sort this issue out properly. And I'm also getting my first Apple Silicon machine this month, so I'll be able to properly try and document the process there too :)
@garyo @ccqpein Thanks for your patience. I have updated #88 and fixed the outstanding issues on it, which was that it did not actually re-link anything after copying in all the relevant shared libraries into the Emacs.app bundle.
If you have time and are willing, feel free to try out the latest changes in #88. My plan is to wrap up and merge it tomorrow if no issues have cropped up with it.
The only catch on Apple Silicon is that you need to run (this is now included in the same PR)codesign --force --deep -s - Emacs.app to ad-hoc self-sign the app. (I'll be automating that soon as a means of resolving #64).
Personally I've tested Emacs builds from it which seem to fully work on completely fresh installs of macOS with only the Xcode Command Line Tools installed. And I've tested it on Apple Silicon, as I finally upgraded from an old Intel-mac last week.
Later this week my plan is to focus on getting the nightly builds repo and homebrew tap updated, along with ARM64 builds for Apple Silicon machines too.