Cross-compile

[joeky888]

This is a handy script but when I execute ./build-zsh-5.8-static -m aarch64 I got this

Building zsh...

  IMAGE=arm64v8/alpine:3.9.5
  KERNEL=linux
  ARCH=aarch64
  CPU=armv8-a
Unable to find image 'arm64v8/alpine:3.9.5' locally
3.9.5: Pulling from arm64v8/alpine
eb93038481dd: Pull complete
Digest: sha256:cae6522b6a351615e547ae9222c9a05d172bc5c3240eec03072d4e1d0429a17a
Status: Downloaded newer image for arm64v8/alpine:3.9.5
standard_init_linux.go:211: exec user process caused "exec format error"

So if I want to compile a arm64 version of zsh-bin, I have to do it on a arm64 machine?

What if the cpu is too weak to do this job?

Well, what I expecting is separating different cpu archs into different files and installing

cross-compilers in dockerfiles, pretty much like

$ ls
Dockerfile.LinuxArm64
Dockerfile.LinuxArm32
Dockerfile.LinuxX86
Dockerfile.LinuxX64
Dockerfile.MingwX64
build-zsh-5.8-static

and then I could build all binaries on a decent machine.

[romkatv]

So if I want to compile a arm64 version of zsh-bin, I have to do it on a arm64 machine?

Correct.

What if the cpu is too weak to do this job?

Patches welcome.

[romkatv]

By the way, why do you want to build it yourself and not use the archive from releases?

[joeky888]

why do you want to build it yourself and not use the archive from releases?

Oh I did. It works so I gave you a github star.

BTW, How do you build all these binaries? QEMU?

[romkatv]

I do arm builds on a raspberry pi. $35 is a bargain. VM would also work.

[romkatv]

Oh, I just realized you are the same person as on that starship issue. Do you mind if I ask why you use starship and not powerlevel10k?

[joeky888]

Because my armV7 android tv box (Android 4.4, no root and developer option is off) doesn't come with git and zsh/bash

and termux requires Android 5.0+.

On a no-root Android 4.4 device the only decent terminal I could use is Pydroid3 which comes with busybox, python and python-pip.

If I want to use powerlevel10k/ohmyzsh/zsh-syntax-highlighting/zsh-autosuggestions without git I couldn't update it regularly, right? I like to keep everything up-to-date. And if I could run zsh, I would install all zsh-plugins I've been used not just p10k.

If starship supports busybox shell then I only need to update the starship binary.

So zsh + git + zsh-plugins vs. starship on a very weak cpu, I would choose starship.

[romkatv]

Thanks for the detailed answer! It's very valuable.

I created zsh-bin pretty much for the use case like yours but I wasn't sure there was demand. To take advantage of all zsh goodies you indeed need git and I intend to build the same kind of static binary for it. The goal is to have full-featured zsh shell anywhere where you have basic posix tools. Busybox should suffice.

[joeky888]

I just wrote a script to cross-compile ncurses and zsh-static for aarch64 systems

apt install -y wget xz-utils make git gcc pkg-config file
apt install -y gcc-aarch64-linux-gnu

export CC_PREFIX="aarch64-linux-gnu"
export LIB_CACHE=$HOME/$CC_PREFIX
export CROSS_COMPILE="$CC_PREFIX-"
export CROSS_COMPILER_PREFIX="$CC_PREFIX-"
export CFLAGS_EXTRA="-I$LIB_CACHE/include"
export LDFLAGS_EXTRA="-static -L$LIB_CACHE/lib64 -L$LIB_CACHE/lib"
cd ~

git clone --depth 1 https://github.com/mirror/ncurses
cd ncurses
./configure --prefix="$LIB_CACHE" --host=$CC_PREFIX --with-build-cc=gcc --without-shared --disable-stripping --enable-widec --without-debug
make
make install # install to $LIB_CACHE
cd ~

wget https://downloads.sourceforge.net/project/zsh/zsh/5.8/zsh-5.8.tar.xz
tar Jxvf zsh-5.8.tar.xz
cd zsh-5.8
./configure --prefix="$LIB_CACHE" --host=$CC_PREFIX --enable-cflags="$CFLAGS_EXTRA" --enable-ldflags="$LDFLAGS_EXTRA"
make
make install # install to $LIB_CACHE

file $LIB_CACHE/bin/zsh

It should output something like

/root/aarch64-linux-gnu/bin/zsh: ELF 64-bit LSB executable, ARM aarch64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=fe4242699fb67baa8e9fc0f7dde23b783b7d4dff, for GNU/Linux 3.7.0, not stripped

Other toolchains you can use are

apt install -y gcc-arm-linux-gnueabi
apt install -y gcc-aarch64-linux-gnu
apt install -y gcc-mips-linux-gnu
apt install -y gcc-mingw-w64-x86-64

You don't have to use rpi or qemu because they are not meant for compiling things, also this script could be integrated into dockerfiles and CI/CD tools so you can build multiple arch binaries at one time.

[romkatv]

This is awesome! I'll try to integrate it into the build script.

[joeky888]

I saw you even built a freebsd binary

which debian/ubuntu/archlinux don't have a proper cross-compiler for it. So either

(A.) compile a cross-compiler yourself or

(B.) use a docker image which already contains a freebsd cross-compiler

If you have any question I would try my best to help you. (I used to cross-compile several softwares)

[joeky888]

I think it is possible to make all zsh files(function scripts, zsh binary and the relocating patch) into a single protable file, just make it a shell script with a binary file embed inside of it.

Like this

#!/bin/sh

##### the relocating patch here ########
# ....
# ....
# ....
##################################

##### function scripts here ############
# ....
# ....
# ....
##################################

##### the zsh binary here #############
begin 755 zsh
M?T5,1$%#$^$&...
end
##################################

When you execute this file, it will extract all functions and scripts into somewhere like /tmp,
then execute the relocating patch and finally the zsh binary itself. This will make it completely portable by executing the relocating patch everytime, what do you think?

I think the anaconda installer and nvidia installer for Linux is doing the same thing(embedding everything into a shell script).

[romkatv]

This will make the zsh-bin worse than a regular zsh. Harder to use and also some scripts can break. It's an important goal that once you install zsh-bin it behaves just like the stock zsh.

[romkatv]

This is awesome! I'll try to integrate it into the build script.

Status update: I gave up on this. Getting cross-compilation to work with musl is not fun at all. More importantly, it doesn't simplify my release workflow. I still need access to a machine (real or virtual) with a suitable CPU to verify that zsh works. I also need to build Zsh on macOS, which is really challenging in cross-compilation model.

So I continue doing native builds. My release script is here: https://github.com/romkatv/zsh-bin/blob/release/mbuild. It does native builds on listed machines over SSH. This is quite flexible as it allows you to use QEMU, Virtual Box, localhost or machines in the cloud to do builds.

[joeky888]

Getting cross-compilation to work with musl is not fun at all

https://musl.cc/ this page includes a lot of cross-toolchains, it's very easy to use

cd ~
wget https://musl.cc/aarch64-linux-musl-cross.tgz
tar zxvf aarch64-linux-musl-cross.tgz

export CC_PREFIX="aarch64-linux-musl"
export SYSROOT="$HOME/$CC_PREFIX-cross"
export LIB_CACHE=$HOME/$CC_PREFIX
export PATH="$SYSROOT/bin:$PATH"
export CFLAGS_EXTRA="-I$LIB_CACHE/include"
export LDFLAGS_EXTRA="-static -L$LIB_CACHE/lib64 -L$LIB_CACHE/lib"
cd ~

git clone --depth 1 https://github.com/mirror/ncurses
cd ncurses
./configure --prefix="$LIB_CACHE" --host=$CC_PREFIX --with-build-cc=gcc --without-shared --disable-stripping --enable-widec --without-debug
make
make install
cd ~

wget https://downloads.sourceforge.net/project/zsh/zsh/5.8/zsh-5.8.tar.xz
tar Jxvf zsh-5.8.tar.xz
cd zsh-5.8
./configure --prefix="$LIB_CACHE" --host=$CC_PREFIX --enable-cflags="$CFLAGS_EXTRA" --enable-ldflags="$LDFLAGS_EXTRA"
make
make install

file $LIB_CACHE/bin/zsh # /root/aarch64-linux-musl/bin/zsh: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, with debug_info, not stripped

Cross-compilation is mostly for embed Linux, even if you build a macOS version of zsh binary, people will still use homebrew to install the latest zsh (brew install zsh is much easier of course)

So from what I've seen on github release pages, developers usually provide only Linux/Windows builds.

to verify that zsh works

I thought the file command is enough for verifying :/

So I continue doing native builds

Oh no this isn't about you, one day you'll get tired and lost interest in this project (like the powerlevel9k developers), and you need people to take over it, you can't expect everyone has a rpi, right? For Linux cross builds you can do it with docker scripts. For macOS builds you can do it with native compilers.

I gave up on this

Years ago I was cross-compiling a python lib which is written in c for my company, It took me a whole week to figure it out (it was 10 times harder than compiling a normal c program). After I left the company I was like "Enough of the Bulls**t". I switched to Go and Rust.

I would like to help, but I am looking for a new job, It is hard to get a job right now due to pandemic.

[romkatv]

Getting cross-compilation to work with musl is not fun at all

https://musl.cc/ this page includes a lot of cross-toolchains, it's very easy to use

Sorry, I gave cross-compilation a try and it's really much more difficult to support than native builds. I'm not signing up for it.

Cross-compilation is mostly for embed Linux

For which CPU?

I'm guessing the overlap between linux systems on which you would like to run zsh with those where you cannot compile zsh is nil.

to verify that zsh works

I thought the file command is enough for verifying :/

Not sure if joking...

So I continue doing native builds

Oh no this isn't about you

Exactly. If cross-compilation is too difficult and time-consuming for me to support, adopting it will make it harder for anyone else to pick up the project. Extra complexity is a barrier for entry.

Honestly, I see very little benefit in cross-compilation but the complexity grows by a ton. These build scripts and patches don't just grow on trees. I need to develop them. To do that I need an environment where I can iterate with turnaround time. Builting infrastructure that would allow this with cross-compilation is a massive undertaking. If it gave something significant in return, that would be one thing. Right now I'm having hard time seeing value. Doing native builds in VMs seems better from any angle.

you can't expect everyone has a rpi

Having access to linux on arm isn't too much to ask.

• you can use a raspberry pi ($30 or $35 if you are rich)
• you can use any android phone
• you can use qemu
• you can use virtual box

Years ago I was cross-compiling a python lib which is written in c for my company, It took me a whole week to figure it out (it was 10 times harder than compiling a normal c program).

Sounds painful. Thanks for sharing.

I would like to help, but I am looking for a new job, It is hard to get a job right now due to pandemic.

Thanks for you help so far.

I'm not asking for your help with cross-compilation. I really don't want to support it.