rust-lang/rust

Linking to private symbols from frameworks between compilation units

madsmtm opened this issue · 16 comments

madsmtm commented

This used to work on Rust nightly-2021-11-24:

// Cargo.toml
[package]
name = "foo"
version = "0.1.0"
edition = "2021"

// lib.rs
#[link(name = "AppKit", kind = "framework")]
extern "C" {}

// main.rs
use foo as _;

#[link(name = "CoreGraphics", kind = "framework")]
extern "C" {}

extern "C" {
    fn CGDisplayCreateUUIDFromDisplayID(display: u32) -> *const std::os::raw::c_void;
}

fn main() {
    unsafe { CGDisplayCreateUUIDFromDisplayID(0) };
}

CGDisplayCreateUUIDFromDisplayID is present in the ColorSync framework (on macOS +10.13; on lower versions, in the CoreGraphics framework). The AppKit framework uses ColorSync internally, so the symbol is available through that (I think).

Linking like this worked previously, but since Rust nightly-2021-11-26 it started failing.

This is affecting winit's CI.

Meta

OS: macOS Mojave 10.14.6.

rustc --version --verbose:

rustc 1.58.0-nightly (dd549dcab 2021-11-25)
binary: rustc
commit-hash: dd549dcab404ec4c7d07b5a83aca5bdd7171138f
commit-date: 2021-11-25
host: x86_64-apple-darwin
release: 1.58.0-nightly
LLVM version: 13.0.0
Backtrace 

   Compiling link_error v0.1.0 (./link_error)
error: linking with `cc` failed: exit status: 1
  |
  = note: "cc" "-m64" "-arch" "x86_64" "./link_error/target/debug/deps/link_error-df545b853490a663.1684psdcp1c8t8w.rcgu.o" "./link_error/target/debug/deps/link_error-df545b853490a663.3mg4gkj495esxxn3.rcgu.o" "./link_error/target/debug/deps/link_error-df545b853490a663.4lqpqqd6m4wsa5ac.rcgu.o" "./link_error/target/debug/deps/link_error-df545b853490a663.4q6v301re75li0ws.rcgu.o" "./link_error/target/debug/deps/link_error-df545b853490a663.4spajtpt70opf5uj.rcgu.o" "./link_error/target/debug/deps/link_error-df545b853490a663.5gpk52ejgwv2r4xx.rcgu.o" "./link_error/target/debug/deps/link_error-df545b853490a663.rk5agyyp0moj2ik.rcgu.o" "./link_error/target/debug/deps/link_error-df545b853490a663.aueco4pi09j3onj.rcgu.o" "-L" "./link_error/target/debug/deps" "-L" "~/.rustup/toolchains/nightly-2021-11-26-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib" "-framework" "CoreGraphics" "./link_error/target/debug/deps/liblink_error-9c8bbc70359362ee.rlib" "~/.rustup/toolchains/nightly-2021-11-26-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libstd-12c85425d8bdabd2.rlib" "~/.rustup/toolchains/nightly-2021-11-26-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libpanic_unwind-c69df909f9b81081.rlib" "~/.rustup/toolchains/nightly-2021-11-26-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libobject-f0c16355db7307b7.rlib" "~/.rustup/toolchains/nightly-2021-11-26-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libmemchr-d69ae56aea540e36.rlib" "~/.rustup/toolchains/nightly-2021-11-26-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libaddr2line-3fd4756c45c77fb7.rlib" "~/.rustup/toolchains/nightly-2021-11-26-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libgimli-1d2aab3cb0cd976a.rlib" "~/.rustup/toolchains/nightly-2021-11-26-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libstd_detect-1491074c150e2040.rlib" "~/.rustup/toolchains/nightly-2021-11-26-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/librustc_demangle-5a7d603578c76924.rlib" "~/.rustup/toolchains/nightly-2021-11-26-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libhashbrown-a5bcb345953a0f7b.rlib" "~/.rustup/toolchains/nightly-2021-11-26-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_alloc-3ffc0cbe23a29292.rlib" "~/.rustup/toolchains/nightly-2021-11-26-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libunwind-ee442041a1b0c90d.rlib" "~/.rustup/toolchains/nightly-2021-11-26-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libcfg_if-e0c705be41bf34ff.rlib" "~/.rustup/toolchains/nightly-2021-11-26-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liblibc-23f0461cd27b5763.rlib" "~/.rustup/toolchains/nightly-2021-11-26-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc-7db0f39aca3e5046.rlib" "~/.rustup/toolchains/nightly-2021-11-26-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_core-7454d006639b0b9c.rlib" "~/.rustup/toolchains/nightly-2021-11-26-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libcore-07b1a016408f5808.rlib" "~/.rustup/toolchains/nightly-2021-11-26-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libcompiler_builtins-852eb1ab762cbb0b.rlib" "-framework" "AppKit" "-lSystem" "-lresolv" "-lc" "-lm" "-liconv" "-L" "~/.rustup/toolchains/nightly-2021-11-26-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib" "-o" "./link_error/target/debug/deps/link_error-df545b853490a663" "-Wl,-dead_strip" "-nodefaultlibs"
  = note: Undefined symbols for architecture x86_64:
            "_CGDisplayCreateUUIDFromDisplayID", referenced from:
                link_error::main::h68dbd3e6b69f3cec in link_error-df545b853490a663.5gpk52ejgwv2r4xx.rcgu.o
          ld: symbol(s) not found for architecture x86_64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

error: could not compile `link_error` due to previous error

madsmtm commented

Sorry if the issue is not very well explained, I'm very inexperienced with how linking works

madsmtm commented

This is affecting coreaudio as well from what I can tell.

ehuss commented

This appears to have regressed via #90499, cc @hkratz.

As a temporary workaround, you can try setting the environment variable MACOSX_DEPLOYMENT_TARGET to something like 11.0.

  • ❤️2
hkratz commented

With #90499 Rust sets MACOSX_DEPLOYMENT_TARGET to 10.7 for the linker, if not set, to match what we are passing to LLVM. Setting it to 10.8 or above get rids of the linker failure.

I will look into this more.

  • 👍1
hkratz commented

This is affecting coreaudio as well from what I can tell.

@madsmtm Can you elaborate on that? Building it locally with cargo +nightly build --target x86_64-apple-darwin worked fine for me on an M1.

madsmtm commented

I can confirm that using MACOSX_DEPLOYMENT_TARGET=10.8 gets rid of the linker failure in winit.
For coreaudio, MACOSX_DEPLOYMENT_TARGET=10.11 is required.

This is affecting coreaudio as well from what I can tell.

@madsmtm Can you elaborate on that? Building it locally with cargo +nightly build --target x86_64-apple-darwin worked fine for me on an M1.

You need to build a binary, so that the linker is actually invoked. Try building an example (e.g. cargo +nightly build --target x86_64-apple-darwin --example feedback).

  • 👍1
cormacrelf commented

I can clarify this a little more: If you were building a Rust program for an Apple platform that linked to APIs with availability somewhere between 10.7+ and 12.0+, and you weren't setting your own deployment target, #90499 has revealed a bug in your build process. Your binaries were only ever going to run on recent OS releases, but you would have been unaware without testing.

For coreaudio, if it really requires APIs from 11.0+, that seems a bit recent/limiting for a library that's existed since forever on macOS. If the authors were unaware of this, #90499 has revealed that they need to do more work to attain backwards compatibility. There will be more breakage like it, but it is the good kind, because anyone using newer Apple APIs should be aware of their minimum platform requirement, just as Xcode forces you to be.

hkratz commented

Hmm, this change had more impact than I thought.

(Open-source) clang does a dance to determine which minimum Macos version to pass as part of the target triple to LLVM and to ld via -platform_version by looking at the various command line options and environment variables that have accumulated over the years. If nothing is specified, it apparently defaults to the minimum of the DefaultDeploymentTargetVersion specified in the SDKROOTs SDKSettings.json and the host Macos version here.

I think ideally we should probably try to mimic this behavior somewhat so we don't confront users with having to set MACOSX_DEPLOYMENT_TARGET, while they don't have to do it using clang. Not sure how we can easily accomplish this though.

madsmtm commented

I can clarify this a little more: If you were building a Rust program for an Apple platform that linked to APIs with availability somewhere between 10.7+ and 12.0+, and you weren't setting your own deployment target, #90499 has revealed a bug in your build process. Your binaries were only ever going to run on recent OS releases, but you would have been unaware without testing.

For coreaudio, if it really requires APIs from 11.0+, that seems a bit recent/limiting for a library that's existed since forever on macOS. If the authors were unaware of this, #90499 has revealed that they need to do more work to attain backwards compatibility. There will be more breakage like it, but it is the good kind, because anyone using newer Apple APIs should be aware of their minimum platform requirement, just as Xcode forces you to be.

Actually, that makes a lot of sense, thanks! I fully agree that it's better to catch these issues when developing (instead of later, when users complain that the application doesn't work on their older system)!

I've opened rust-windowing/winit#2078 to fix the linking in winit, and RustAudio/coreaudio-sys#49 would fix the linking in coreaudio.

madsmtm commented

Note that this is a breaking change for macOS users, before, they didn't have to specify MACOSX_DEPLOYMENT_TARGET, now they do. Furthermore, they have no easy way to find out that they need to (they just get a confusing linker error)!

Additionally, the affected crates (that we know about) are quite widely used (at least half of all Rust games depend on winit and/or coreaudio-sys), and have not yet fixed the issue, nor yet released updated versions which includes these fixes. And even when this has been done, it will take time before users have run cargo update and received working versions (can probably be helped along by yanking the affected versions).

I don't really know the procedure for this, but I would like to propose reverting #90499 (at least from the current beta) to give the affected crates more time to update.

Again, I really do agree with the change, I just also want to minimize the breakage macOS users will experience from upgrading Rust.

hkratz commented

Like I wrote above I think ideally we should match clang's behavior and thus fall back to the default of the currently configured Macos SDK. That would mean looking at the Version property of the SDKSettings.json in the currently configured SDK. Unfortunately I don't really see a reliable way to do get the path to that without running xcrun --show-sdk-path.

jyn514 commented

I think ideally we should probably try to mimic this behavior

@hkratz I think we should revert the change until you find a way to do this - it seems to be causing a lot of breakage and my understanding is the goal was just to turn a runtime error into a link time error.

hkratz commented

@jyn514 Oh sure. I am not against a revert until that is sorted out. I will open a PR.

@rustbot claim

  • ❤️2
pnkfelix commented

reopening as this is a beta-regression, and we keep those open until beta fix lands (or we explicitly decline to fix).

pietroalbini commented

Backport merged.

  • ❤️1
madsmtm commented

Just for reference, the linking in the noted crates have since been fixed: