TimNN/cargo-lipo

Linker errors when building with --xcode-integ on Big Sur

irh opened this issue · 5 comments

irh commented

I'm running into build errors on Big Sur when using cargo-lipo in an Xcode build script.

I've made a small test project here.

It builds successfully in Xcode 12.2 on Catalina 10.15.7, but fails with Big Sur 11.0.1.

The errors are along the lines of

linking with cc failed
note: ld: library not found for -lSystem

Building the project on the command line with cargo lipo without --xcode-integ works correctly.

Big Sur has introduced a dynamic library cache, so libSystem.dylib isn't in /usr/lib as it is on Catalina, I assume that's got something to do with it but I haven't had any luck finding a solution so far.

Any ideas? Thanks for cargo-lipo 👍

v-almonacid commented

@irh I was drafting an issue at exactly the same time 😄 Good to know I'm not alone on this. I still don't have a good solution for this but I'll share what I tried:

First, looking at the linker (ld) docs, they say the libs are searched in /usr/lib and then /usr/local/lib. ld doesn't provide a command to check what are the actual search paths, but you can get it from gcc -Xlinker -v, which in my case shows:

Library search paths:
	/usr/local/lib
	/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib

This was my first "bingo!". They removed the standard libs from /usr/lib and ld is no longer looking there (probably this is a result of their new "dynamic linker cache" which have broken so much stuff and produced so much pain). Anyway, I think the issue is that when we build for iOS the linker must be looking in /Applications/Xcode.app/.../SDKs/*iPhoneOS.platform*/usr/lib instead, which in my case doesn't contain the libs.

As a workaround, I tried copying the missing library symlinks in usr/local/lib, where we know the linker is looking. So:

  cd /usr/local/lib 
  sudo ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib/libSystem.B.tbd ./libSystem.tbd
  sudo ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib/libresolv.9.tbd ./libresolv.tbd
  sudo ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib/libSystem.tbd ./libc.tbd
  sudo ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib/libc++.tbd ./libc++.tbd
  sudo ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib/libSystem.tbd ./libm.tbd

This works on the simulator only (probably because I'm using the MacOSX sdk and not the iPhoneOS one. The question is why there are no standard C libs in /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/usr/lib/.

I hope this helps and if you find out something else please report here! Also I really hope @TimNN can take a look, he'll probably will understand the issue much better than I.

UPDATE: I just tried this hack again using a real iphone and it worked. They only thing that changed wrt the last time I tried is that I upgraded my command line tools yesterday.

  • 👍2
  • ❤️2
TimNN commented

@v-almonacid: Thanks for the vote of confidence, but I haven’t touched or used this code in years.

I‘d be happy to review any patches related to this, but probably won’t spend any time investigating this in the near future. (I‘ll see about updating the readme to make the maintenance status of the project clear).

irh commented

@v-almonacid Thanks for the pointers. I haven't fully cracked this yet but I made some progress.

Rather than trying to force the use of /usr/lib via symlinking, adding the following to my test build script before the call to cargo lipo fixes the linking issue for me.

SDKROOT=`xcrun --sdk macosx --show-sdk-path`
export LIBRARY_PATH="$SDKROOT/usr/lib"

While poking at this I found that building directly with cargo has the same issue, and found that adding

export SDKROOT=`xcrun --sdk macosx --show-sdk-path`

before the call to cargo is enough to fix the issue. This doesn't work for me with cargo lipo however, where it's necessary to instead set LIBRARY_PATH.

@TimNN Thanks for clarifying, I'm not at the point of making a PR but it's good to know that it would be accepted.

  • 👍3
ubamrein commented

We had the exact same issue, but we found out that there are two "linkers":

  • The one in the Xcode.app bundle
  • /usr/bin/clang

both report the same version AND the same "InstalledDir" since /usr/bin/clang in the end probably executes the one in the xcode.app bundle (it is not a symlink though):

> /usr/bin/clang --version
Apple clang version 12.0.0 (clang-1200.0.32.28)
Target: x86_64-apple-darwin20.2.0
Thread model: posix
InstalledDir: /Applications/Xcode_12_3.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Linking with /usr/bin/clang works, so we hardcoded the linker in the cargo config file as following:

[target.aarch64-apple-ios]
linker="/usr/bin/clang"
[target.x86_64-apple-darwin]
linker="/usr/bin/clang

We were not explicitly using cargo lipo, but I found this issue and thought I dump our experience as well :)

Cldfire commented

I stumbled across a neat workaround for this here:

if [[ -n "${DEVELOPER_SDK_DIR:-}" ]]; then
  # Assume we're in Xcode, which means we're probably cross-compiling.
  # In this case, we need to add an extra library search path for build scripts and proc-macros,
  # which run on the host instead of the target.
  # (macOS Big Sur does not have linkable libraries in /usr/lib/.)
  export LIBRARY_PATH="${DEVELOPER_SDK_DIR}/MacOSX.sdk/usr/lib:${LIBRARY_PATH:-}"
fi

This worked great for me on my machine and is a simpler solution than some of the other options presented in this thread.

  • 👍6