Support running natively on aarch64 / arm64 hosts / Mac M1/M2

Question

Support running natively on aarch64 / arm64 hosts / Mac M1/M2

Opened this issue 2 years ago · 11 comments

Checklist

I've looked through the issues and pull requests for similar request
This feature could be solved with a custom docker image (optional)

Describe your request

With the rise of arm64 architecture, cross should natively support this host architecture.

This means that the docker images should run on aarch64, and that cross should not make assumptions that the host is x86_64.

Describe why this would be a good inclusion for `cross`

No response

Answer 1 · 2022-06-03T16:10:58.000Z

prior work: #567

Answer 2 · 2022-06-03T16:12:57.000Z

One issue with this is that as mentioned, ubuntu does not ship the same packages for arm64 as it does for x86_64.

Another issue is how we should solve building the images. I think we could use buildx, but that would severely increase CI time I think

Answer 3 · 2022-07-23T23:38:13.000Z

as of #817 we can now let consumers create a custom image, built for aarch64, and mount the aarch64 toolchain.

# Cross.toml
[target.x86_64-unknown-linux-gnu]
pre-build = [
    # Install the cross compiler
    "apt-get update && apt-get install -y libc6 g++-x86-64-linux-gnu libc6-dev-amd64-cross gcc",
]

[target.x86_64-unknown-linux-gnu.env]
# Set the linker and information for `cc` crate to link properly
passthrough = [
    "CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_LINKER=x86_64-linux-gnu-gcc",
    "CC_x86_64_unknown_linux_gnu=x86_64-linux-gnu-gcc",
    "CXX_x86_64_unknown_linux_gnu=x86_64-linux-gnu-g++",
]

[target.x86_64-unknown-linux-gnu.image]
name = "ubuntu:20.04"
# This is the magic, if left unspecified, the default would be that the toolchain is `x86_64-unknown-linux-gnu`
toolchain = ["aarch64-unknown-linux-gnu"]

cross build --target x86_64-unknown-linux-gnu -v
+ cargo metadata --format-version 1 --filter-platform x86_64-unknown-linux-gnu
+ rustc --print sysroot
+ /usr/local/bin/docker
+ /usr/local/bin/docker version '-f "{{ .Server.Os }},{{ .Server.Arch }}"'
+ rustup --verbose toolchain list
+ rustup --verbose target list --toolchain stable-aarch64-unknown-linux-gnu
+ rustup --verbose component list --toolchain stable-aarch64-unknown-linux-gnu
+ "/Users/emil/workspace/cross-tests/basic" /usr/local/bin/docker build --platform linux/arm64 --label 'org.cross-rs.for-cross-target=x86_64-unknown-linux-gnu' --label 'org.cross-rs.runs-with=aarch64-unknown-linux-gnu' --label 'org.cross-rs.workspace_root=/Users/emil/workspace/cross-tests/basic' --tag cross-custom-basic:x86_64-unknown-linux-gnu-50dfa-pre-build --build-arg 'CROSS_CMD=apt-get update && apt-get install -y libc6 g++-x86-64-linux-gnu libc6-dev-amd64-cross' --build-arg 'CROSS_DEB_ARCH=amd64' --file /Users/emil/workspace/cross-tests/basic/target/x86_64-unknown-linux-gnu/Dockerfile.x86_64-unknown-linux-gnu-custom .
[+] Building 10.0s (6/6) FINISHED                                                                               
 => [internal] load build definition from Dockerfile.x86_64-unknown-linux-gnu-custom                       0.0s
 => => transferring dockerfile: 166B                                                                       0.0s
 => [internal] load .dockerignore                                                                          0.0s
 => => transferring context: 2B                                                                            0.0s
 => [internal] load metadata for docker.io/library/ubuntu:20.04                                            0.6s
 => CACHED [1/2] FROM docker.io/library/ubuntu:20.04@sha256:fd92c36d3cb9b1d027c4d2a72c6bf0125da82425fc2ca  0.0s
 => [2/2] RUN eval "apt-get update && apt-get install -y libc6 g++-x86-64-linux-gnu libc6-dev-amd64-cross  8.8s
 => exporting to image                                                                                     0.5s 
 => => exporting layers                                                                                    0.5s 
 => => writing image sha256:92f3cf51cc16c7b176c9bc09ce28ef83c029881e9c5017589eb8372bf6fff8d3               0.0s 
 => => naming to docker.io/library/cross-custom-basic:x86_64-unknown-linux-gnu-50dfa-pre-build             0.0s 
+ /usr/local/bin/docker run --userns host --platform linux/arm64 -e 'CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_LINKER=x86_64-linux-gnu-gcc' -e 'CC_x86_64_unknown_linux_gnu=x86_64-linux-gnu-gcc' -e 'CXX_x86_64_unknown_linux_gnu=x86_64-linux-gnu-g++' -e 'PKG_CONFIG_ALLOW_CROSS=1' -e 'XARGO_HOME=/xargo' -e 'CARGO_HOME=/cargo' -e 'CARGO_TARGET_DIR=/target' -e 'CROSS_RUNNER=' -e TERM -e 'USER=emil' --rm --user 501:20 -v /Users/emil/.xargo:/xargo:Z -v /Users/emil/.cargo:/cargo:Z -v /cargo/bin -v /Users/emil/workspace/cross-tests/basic:/project:Z -v /Users/emil/.rustup/toolchains/stable-aarch64-unknown-linux-gnu:/rust:Z,ro -v /Users/emil/workspace/cross-tests/basic/target:/target:Z -w /project -i -t cross-custom-basic:x86_64-unknown-linux-gnu-50dfa-pre-build sh -c 'PATH=$PATH:/rust/bin cargo build --target x86_64-unknown-linux-gnu -v'
       Fresh basic v0.1.0 (/project)
    Finished dev [unoptimized + debuginfo] target(s) in 0.47s

basic on master [?] is 📦 v0.1.0 via 🦀 v1.61.0 took 12s 
❯ file target/x86_64-unknown-linux-gnu/debug/basic
target/x86_64-unknown-linux-gnu/debug/basic: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=5716ca6c6514e255e9a29eeeeac3d9cb4fb43c0a, for GNU/Linux 3.2.0, with debug_info, not stripped

basic on master [?] is 📦 v0.1.0 via 🦀 v1.61.0 
❯ uname -a
Darwin Emils-MacBook-Pro.local 21.5.0 Darwin Kernel Version 21.5.0: Tue Apr 26 21:08:37 PDT 2022; root:xnu-8020.121.3~4/RELEASE_ARM64_T6000 arm64

Answer 4 · 2022-07-27T17:18:19.000Z

As for providing images natively, we'd likely have to have separately images for aarch64 and x86_64. Attempting to say, build x86_64-unknown-linux-gnu for ~~linux/arm64/v8=x86_64-unknown-linux-gnu~~ has issues due to missing installation candidates for gcc-multilib. This package is only provided for x86, x86_64, and s390x, so we'd need to likely maintain a separate image to cross-compiler to x86_64 here (probably gcc-multilib-x86-64-linux-gnu).

EDIT: Apparently that's the wrong way to go about it.

Answer 5 · 2022-07-27T17:21:45.000Z

why are you mounting the x86_64 toolchain on arm64?

It should be linux/arm64/v8=aarch64-unknown-linux-gnu

Answer 6 · 2022-07-27T17:29:37.000Z

why are you mounting the x86_64 toolchain on arm64?

It should be linux/arm64/v8=aarch64-unknown-linux-gnu

So I can build a cross-compiler to x86_64 for an aarch64 host that doesn't require Qemu emulation to run the image.

Answer 7 · 2022-07-27T17:51:25.000Z

For transparency, the correct invocation for building a aarch64 image for a specific target is

cargo build-docker-image --platform=arch64-unknown-linux-gnu <target>

Answer 8 · 2022-07-28T06:29:22.000Z

We're tracking the progress of aarch64 images in #975

Answer 9 · 2023-02-15T18:12:21.000Z

I'm attempting to cross-compile from m1 mac to a raspberry pi. I believe both are aarch64 machines.
Is the correct approach to do something like this?

[build]
default-target = "aarch64-unknown-linux-gnu"

[target.aarch64-unknown-linux-gnu.image]
name = "docker.io/thedevminertv/slimevr-testing-jig-buildbox:arm64"
# toolchain = ["aarch64-unknown-linux-gnu"] # Uncomment if you are on aarch64 mac

Answer 10 · 2023-02-15T20:52:10.000Z

You can specify all the architectures the image works for and it'll try to be smart about it.

[target.aarch64-unknown-linux-gnu.image]
name = "my-multi-arch-image:tag"
toolchain = ["aarch64-unknown-linux-gnu", "x86_64-unknown-linux-gnu"]

Answer 11 · 2023-10-24T08:31:02.000Z

To also install :amd64 packages, you'll need to add the non-ports sources

[target.x86_64-unknown-linux-gnu]
pre-build = [
    "apt-get update && apt-get install -y libc6 g++-x86-64-linux-gnu libc6-dev-amd64-cross gcc",
    "sed 's/^deb http/deb [arch=arm64] http/' -i '/etc/apt/sources.list'",
    "echo 'deb [arch=amd64] http://security.ubuntu.com/ubuntu/ focal-security main restricted universe multiverse' >> /etc/apt/sources.list",
    "echo 'deb [arch=amd64] http://archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse' >> /etc/apt/sources.list",
    "echo 'deb [arch=amd64] http://archive.ubuntu.com/ubuntu/ focal-updates main restricted universe multiverse' >> /etc/apt/sources.list",
    "echo 'deb [arch=amd64] http://archive.ubuntu.com/ubuntu/ focal-backports main restricted universe multiverse' >> /etc/apt/sources.list",
    "dpkg --add-architecture amd64",
    "apt-get update && apt-get install -y libssl-dev:amd64",
]

[target.x86_64-unknown-linux-gnu.env]
passthrough = [
    "CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_LINKER=x86_64-linux-gnu-gcc",
    "CC_x86_64_unknown_linux_gnu=x86_64-linux-gnu-gcc",
    "CXX_x86_64_unknown_linux_gnu=x86_64-linux-gnu-g++",
]

[target.x86_64-unknown-linux-gnu.image]
name = "ubuntu:20.04"
toolchain = ["aarch64-unknown-linux-gnu"]

Checklist

Describe your request

Describe why this would be a good inclusion for cross

Describe why this would be a good inclusion for `cross`