[FR] macOS & Apple Silicon Support

Question

[FR] macOS & Apple Silicon Support

jonpryor opened this issue 4 years ago · 43 comments

At WWDC 2020, Apple announced that macOS will be migrating to "Apple Silicon" over the next couple of years. While Rosetta 2 will exist as an immediate solution to allow the execution of existing x86_64 code on devices with Apple Silicon CPUs, the mid-to-long term plan should be to distribute all native binaries as Universal 2 binaries: executables which contain both x86_64 and Apple Silicon slices:

https://developer.apple.com/documentation/xcode/building_a_universal_macos_binary

All native binaries in the Android SDK & NDK, such as clang and the linker, presumably need to be built as Universal 2 binaries.

Are there any plans to distribute the NDK as Universal 2 binaries?

Is your feature request related to a problem? Please describe.

Apple will be migrating macOS to a new hardware architecture. It would be nice if it were properly supported before x86 support is dead. (Admittedly not "soon"…)

Describe the solution you'd like

Ship NDK binaries as Universal 2 binaries, containing both x86_64 and Apple Silicon support.

Describe alternatives you've considered

Crying.

Components that need to be made fat

Won't fix:

GDB
GAS
renderscript

Answer 1 · 2020-06-26T22:48:23.000Z

Thanks for filing. We've been planning for this since the rumors started being published, but hadn't filed a tracking bug.

We're working on getting some of the development devices. We also need to get buildbots that have a new enough xcode (which could possibly require a new version of macOS, I don't know). Some test infra work probably needs to be done, since I'm assuming that the new OS will be cross compiled for rather than natively compiled for.

After we have all the infrastructure we can finally start actually working on this. There may be some porting work, but for the most part we're shipping third-party software and will be dependent on those projects being ready.

In other words, there's a pretty long tail of work here, but we have started on it. There's still a non-zero chance that we'll need to rely on Rosetta at first. There is an outstanding question of whether or not Rosetta will support bare executables or just app bundles.

Crying.

Same.

Answer 2 · 2020-06-28T08:38:03.000Z

The chances are pretty high that x86_64 NDK will run decently in Rosetta-2 (translation) mode even on the ARM hardware.

Answer 3 · 2020-06-30T16:56:48.000Z

We can confirm that the x86_64 NDK tools that Xamarin.Android uses run on Rosetta 2 on the Apple DTK. I can't say how "well" they run, but they do run, and we're able to use Rosetta 2-translated SDK & NDK utilities to build & install a Xamarin.Android application.

Answer 4 · 2020-06-30T20:01:10.000Z

Great, thanks for that info. Just to check, they run outside your app bundle as well?

Answer 5 · 2020-07-02T17:30:55.000Z

Great, thanks for that info. Just to check, they run outside your app bundle as well?

No, we re-package and redistribute bits of the Android NDK toolchain such that they are inside our .pkg, e.g.:

/Library/Frameworks/Xamarin.Android.framework/Versions/10.2.0.100/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/ndk/aarch64-linux-android-ld

I don't have the DTK machine, so I can't verify that they run outside of our default installation location. I don't see why they wouldn't…

Answer 6 · 2020-07-06T09:36:32.000Z

I would hope they do as well, but so far I haven't seen anything definitive about whether Rosetta will work for plain executables outside an app package. Our recent experience with notarization has me worried that it won't.

We should have our hands on the hardware soon enough and be able to find out though.

Answer 7 · 2020-12-14T20:09:13.000Z

going back to the original question:

Are there any plans to distribute the NDK as Universal 2 binaries?

we're still thinking about whether we'll have universal binaries or two separate NDKs for darwin/x86-64 and darwin/arm64, so if you have a strong argument why you'd rather have universal binaries than a separate NDK, let us know while we/Studio are still trying to decide! we don't have any timeline to announce yet, but we have started work on building everything for darwin/arm64. (and in the meantime, everything works with Rosetta2.)

Answer 8 · 2020-12-15T19:52:05.000Z

If you ask me, I'd prefer the smaller separate packages. These days, we mostly download NDK via SDK Manager, and it should know well which flavor of NDK is correct for the given machine.

BTW, same applies to Windows and Linux.

Answer 9 · 2020-12-15T22:31:03.000Z

(and in the meantime, everything works with Rosetta2.)

How do you run ndk using rosetta2?

Answer 10 · 2020-12-15T22:35:18.000Z

Nothing special I don't think. The whole point of Rosetta is to allow binaries to run as if nothing were different.

Answer 11 · 2020-12-15T22:56:59.000Z

yeah, that's my understanding too (and how it was for "classic" Rosetta back in the PowerPC transition days). @kongy has actually tried it...

Answer 12 · 2020-12-29T19:01:20.000Z

hi, I get this error while trying to build a project with ndk

Error while executing process /Users/osrl/Library/Android/sdk/ndk/21.3.6528147/ndk-build with arguments {NDK_PROJECT_PATH=null APP_BUILD_SCRIPT=/Users/osrl/Documents/Android Studio/hesabim-android/app/src/main/jni/Android.mk NDK_APPLICATION_MK=/Users/osrl/Documents/Android Studio/hesabim-android/app/src/main/jni/Application.mk APP_ABI=armeabi-v7a NDK_ALL_ABIS=armeabi-v7a NDK_DEBUG=1 APP_PLATFORM=android-18 NDK_OUT=/Users/osrl/Documents/Android Studio/hesabim-android/app/build/intermediates/ndkBuild/debug/obj NDK_LIBS_OUT=/Users/osrl/Documents/Android Studio/hesabim-android/app/build/intermediates/ndkBuild/debug/lib APP_SHORT_COMMANDS=false LOCAL_SHORT_COMMANDS=false -B -n}
ERROR: Unknown host CPU architecture: arm64

I also tried to build with arch -x86_64 but no luck. How does "everything work with Rosetta2"? Can you help @enh-google?

Answer 13 · 2020-12-30T19:37:21.000Z

I solved it by editing ndk-build script:

#!/bin/sh
DIR="$(cd "$(dirname "$0")" && pwd)"
arch -x86_64 /bin/bash $DIR/build/ndk-build "$@"

After that, ndk-build gave me "invalid option -- O" error
I edited build/ndk-build script and removed -O argument from the command:
$GNUMAKE -f $PROGDIR/core/build-local.mk "$@

Can anyone tell me if I did something wrong? Why did I had to edit ndk-build?

Answer 14 · 2021-01-05T22:20:14.000Z

Presumably you're getting your system's make and not the NDK's when you make that change. If you're using an older make there are going to be some broken parts of ndk-build.

Answer 15 · 2021-01-05T22:29:16.000Z

(i don't have a macOS 11 machine to check, but 10.15 certainly only had make 3.81, which is too old to have -O, and i don't expect Apple to have updated to a GPL3 version of GNU Make...)

Answer 16 · 2021-02-10T11:25:09.000Z

I solved it by editing ndk-build script:
#!/bin/sh
DIR="$(cd "$(dirname "$0")" && pwd)"
arch -x86_64 /bin/bash $DIR/build/ndk-build "$@"
After that, ndk-build gave me "invalid option -- O" error
I edited build/ndk-build script and removed -O argument from the command:
$GNUMAKE -f $PROGDIR/core/build-local.mk "$@

Can anyone tell me if I did something wrong? Why did I had to edit ndk-build?

Thanks

Answer 17 · 2021-03-03T18:12:44.000Z

(i don't have a macOS 11 machine to check, but 10.15 certainly only had make 3.81, which is too old to have -O, and i don't expect Apple to have updated to a GPL3 version of GNU Make...)

My macOS 11.2.1 machine with Xcode 12.2 still has GNU make 3.81.

Answer 18 · 2021-04-09T10:18:33.000Z

Do you have Rosetta 2 installed? That will be needed to run the compiler, which is an Intel binary. Rosetta 2 isn't always installed automatically. John

…

On Fri, Apr 9, 2021 at 7:14 AM H3c ***@***.***> wrote: how to build use cmake??? i get some error on M1 C/C++: /Users/xxx/Documents/THAILib/app/CMakeLists.txt debug|armeabi-v7a : CMake Error at /Users/xxx/Library/Android/sdk/cmake/3.18.1/share/cmake-3.18/Modules/CMakeTestCCompiler.cmake:66 (message): The C compiler "/Users/xxx/Documents/android-ndk-r21e/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang" is not able to compile a simple test program. It fails with the following output: Change Dir: /Users/xxx/Documents/THAILib/app/.cxx/Debug/3t4w4qs6/armeabi-v7a/CMakeFiles/CMakeTmp Run Build Command(s):/Users/xxx/Library/Android/sdk/cmake/3.18.1/bin/ninja cmTC_ef1bc && [1/2] Building C object CMakeFiles/cmTC_ef1bc.dir/testCCompiler.c.o FAILED: CMakeFiles/cmTC_ef1bc.dir/testCCompiler.c.o /Users/xxx/android-ndk-r21e/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang --target=armv7-none-linux-androideabi21 --gcc-toolchain=/Users/h3c/Documents/android-ndk-r21e/toolchains/llvm/prebuilt/darwin-x86_64 --sysroot=/Users/xxx/Documents/android-ndk-r21e/toolchains/llvm/prebuilt/darwin-x86_64/sysroot -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -march=armv7-a -mthumb -Wformat -Werror=format-security -fPIE -MD -MT CMakeFiles/cmTC_ef1bc.dir/testCCompiler.c.o -MF CMakeFiles/cmTC_ef1bc.dir/testCCompiler.c.o.d -o CMakeFiles/cmTC_ef1bc.dir/testCCompiler.c.o -c testCCompiler.c ninja: build stopped: subcommand failed. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1299 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ATCLA5X3RCLVXMCOIX25UKLTH2LNPANCNFSM4OJWV7QA> .

Answer 19 · 2021-06-09T23:37:02.000Z

I solved it by editing ndk-build script:
#!/bin/sh
DIR="$(cd "$(dirname "$0")" && pwd)"
arch -x86_64 /bin/bash $DIR/build/ndk-build "$@"
After that, ndk-build gave me "invalid option -- O" error
I edited build/ndk-build script and removed -O argument from the command:
$GNUMAKE -f $PROGDIR/core/build-local.mk "$@

Can anyone tell me if I did something wrong? Why did I had to edit ndk-build?

same problem

Answer 20 · 2021-07-26T20:58:22.000Z

Enumerated what I think are all the subtasks here and filed bugs for them (task list in the OP). LLVM is ~done (we just need to take the latest version), but iirc someone told me that a Rosetta process invoking a fat binary will result in another Rosetta process, so until CMake/ninja/make are updated that doesn't actually change anything (though it does for other build systems).

Answer 21 · 2021-08-03T10:53:47.000Z

Enumerated what I think are all the subtasks here and filed bugs for them (task list in the OP). LLVM is ~done (we just need to take the latest version), but iirc someone told me that a Rosetta process invoking a fat binary will result in another Rosetta process, so until CMake/ninja/make are updated that doesn't actually change anything (though it does for other build systems).

Hi, aren't CMake, make, Ninja and yasm available for M1? At least the homebrew versions seem to be.
https://formulae.brew.sh/formula/make
https://formulae.brew.sh/formula/cmake
https://formulae.brew.sh/formula/ninja
https://formulae.brew.sh/formula/yasm

Answer 22 · 2021-08-03T17:04:56.000Z

You're welcome to use those. We need to update the ones that we build ourselves.

Answer 23 · 2021-08-11T17:07:44.000Z

You're welcome to use those.

How can we tell the NDK to use the "native" brew versions? Is this trivial or does it require digging into the internals. In general I would be fine to dig into the internals but it doesn't seem worth it since it's already "working", but slower

Answer 24 · 2021-09-02T06:45:14.000Z

I assume we also want M1 versions of the shader-tools binaries?

Answer 25 · 2021-09-02T06:55:18.000Z

Good catch. Filed a bug for that.

Answer 26 · 2021-11-29T02:40:58.000Z

I solved it by editing ndk-build script:
#!/bin/sh
DIR="$(cd "$(dirname "$0")" && pwd)"
arch -x86_64 /bin/bash $DIR/build/ndk-build "$@"
After that, ndk-build gave me "invalid option -- O" error I edited build/ndk-build script and removed -O argument from the command: $GNUMAKE -f $PROGDIR/core/build-local.mk "$@

Can anyone tell me if I did something wrong? Why did I had to edit ndk-build?

I added the above configuration, but running ndk-build still returns the error: ERROR: Unknown host CPU architecture: arm64. Is there something I don't set?

Answer 27 · 2021-11-29T20:16:05.000Z

I think #1410 is the thread you want. The fix is in the next release, and https://android-review.googlesource.com/c/platform/ndk/+/1855814 was the fix (in the other thread, but buried in the middle).

Answer 28 · 2021-12-15T12:27:20.000Z

How long is this still going to take? Its been over a year now.. @google wtf ist wrong with you??

Answer 29 · 2021-12-15T12:35:47.000Z

The fix is in the next release

@DanAlbert Do you know what the next is ? At least it's not 23.1.7779620

Answer 30 · 2021-12-15T17:55:34.000Z

The fix is in the next release

@DanAlbert Do you know what he next is ? At least it's not 23.1.7779620

you can see on any given bug... if you look at #1410, say, you can see on the right-hand side that it's currently targeted at r23c and r24.

for dates for any given release, see https://github.com/android/ndk/wiki#release-schedule

Answer 31 · 2021-12-15T19:16:23.000Z

Most of the bugs are still open because they aren't fixed in every supported release. The latest r24 beta has nearly complete M1 support. r23b shipped M1 support for the toolchain, and r23c will include the same level of M1 support that's currently in r24. I'm hoping to release r23b in January but we're still waiting for toolchain fixes before that ships.

The only things that aren't fixed in r24 are yasm and simpleperf. Yasm I suspect no one actually uses so isn't a high priority, and simpleperf is out of our hands.

The big remaining components are CMake and Ninja, which are not a part of the NDK so are not tied to any release. New releases for those are being tested as I write this, but there are some infrastructural questions we need to answer about the release process before they ship (we have not been the team that released those in the past, and there is always something to learn during a handoff).

Answer 32 · 2021-12-15T19:24:08.000Z

The only things that aren't fixed in r24 are yasm and simpleperf. Yasm I suspect no one actually uses so isn't a high priority, and simpleperf is out of our hands.

(by which he means "out of the NDK team's hands" ... obviously simpleperf is "ours" in the wider sense of "Android's" too, and the relevant people are working on that. but simpleperf comes from different infrastructure again from any of the other things mentioned here, so that's a third team that has work to do because of this.)

Answer 33 · 2021-12-17T11:02:52.000Z

Hello @jinanzhuan . You need to edit two files:

one Library/Android/sdk/ndk/21.3.6528147/ndk-build

#!/bin/sh
DIR="$(cd "$(dirname "$0")" && pwd)"
arch -x86_64 /bin/bash $DIR/build/ndk-build "$@"

other one Library/Android/sdk/ndk/21.3.6528147/build/ndk-build
edit this line (it's at the end of the file):

$GNUMAKE -f $PROGDIR/core/build-local.mk "$@

Note that this is a workaround with rosetta.

Answer 34 · 2021-12-17T11:20:21.000Z

https://stackoverflow.com/a/69540773/1079990 solution A without editing a file or solution C

Answer 35 · 2022-01-12T00:30:04.000Z

(Don't read too much into the move from r24 from r24b. Progress was made in r24, the tracking bug is just always going to be moving forward until this is 100%)

Answer 36 · 2022-01-12T02:45:43.000Z

Hello @jinanzhuan . You need to edit two files:

one Library/Android/sdk/ndk/21.3.6528147/ndk-build
#!/bin/sh
DIR="$(cd "$(dirname "$0")" && pwd)"
arch -x86_64 /bin/bash $DIR/build/ndk-build "$@"
other one Library/Android/sdk/ndk/21.3.6528147/build/ndk-build edit this line (it's at the end of the file):
$GNUMAKE -f $PROGDIR/core/build-local.mk "$@
Note that this is a workaround with rosetta.

Thank you for your help! I solved it by restoring the mac system.

Answer 37 · 2022-01-12T17:53:05.000Z

Please take the ndk-build discussion to the appropriate thread: #1410

Answer 38 · 2022-02-01T00:10:12.000Z

There is a new CMake 3.22.1 package in the SDK Manager's canary channel that has M1 support for macOS.

With this release, and with the latest platform-tools and build-tools packages, it's now possible to build and debug an Android app with a CMake project, on an M1 Mac without Rosetta (but with beta/canary packages). These versions of components are needed:

Android Studio 2021.1.1
Android Gradle Plugin 7.1.0 (the plugin includes an aapt2 binary)
Android SDK Build-Tools 32.1-rc1 (SDK's stable channel)
Android SDK Platform Tools 32.0.0 (SDK's stable channel)
Android NDK 24.0.7956693 (beta 2, from the SDK's beta channel)
CMake 3.22.1 (SDK's canary channel)

Android Studio can be briefly configured to use the canary channel to install just CMake, then switched back to the stable channel, to avoid updating everything else.

The NDK and CMake versions need to be configured in build.gradle, e.g.:

android {
    ndkVersion "24.0.7956693-beta2"
    externalNativeBuild {
        cmake {
            path file('src/main/cpp/CMakeLists.txt')
            version '3.22.1'
        }
    }
}

In NDK r24 beta 2, yasm and simpleperf are still x86_64-only.

Using ndk-build on a non-Rosetta M1 Mac needs a newer NDK r24 build, which has a fix to the makefiles to use a universal Python3 binary. The NDK r24 final release should have this fix.

Answer 39 · 2022-04-06T09:34:48.000Z

I solved it by editing ndk-build script:
#!/bin/sh
DIR="$(cd "$(dirname "$0")" && pwd)"
arch -x86_64 /bin/bash $DIR/build/ndk-build "$@"
After that, ndk-build gave me "invalid option -- O" error
I edited build/ndk-build script and removed -O argument from the command:
$GNUMAKE -f $PROGDIR/core/build-local.mk "$@
Can anyone tell me if I did something wrong? Why did I had to edit ndk-build?
Thanks

am having this issue now. how did you get the ndk installed to modify it?

Answer 40 · 2022-04-06T19:19:48.000Z

Locking the thread to encourage folks to use the sub-topics for each component. We'll use this one for announcements about M1 support in general. Please use the appropriate sub-topic (see the list of all components in the OP) if you have questions about M1 support for a specific component, or file a new bug if there's a component we've missed.

Answer 41 · 2022-05-10T22:19:41.000Z

As before, don't worry about the fact that this issue was removed from r23c. r23c still contains improvements for M1, but not all the cherry-picks are safe to backport so it's being moved because r23c won't get all the fixes. The only remaining NDK components in r23c that will not be universal binaries are:

yasm
simpleperf
GAS (won't fix)
GDB (won't fix)
renderscript (won't fix)

As noted in the main post (though I'd previously forgotten to mention renderscript) the "won't fix" components won't be fixed because they're gone in r24+ anyway.

Answer 42 · 2022-06-27T23:48:00.000Z

Same as the last update: just moving things forward as releases move forward. Only affected NDK components in r25 are simpleperf and yasm. Will consider backports to r25 for each once we have fixes available, but for now triaging as r26. r25 is already complete for the vast majority of workflows.

Answer 43 · 2022-07-22T01:39:00.000Z

This is now done enough that if you're on up to date tools you likely won't notice the one remaining piece (yasm). Yasm won't be landing for a while since the next major release is next year. If we've overlooked something that makes yasm way more important we believe it is, please chime in on #1549.