arm64 arch support
NoxInmortus opened this issue · 12 comments
Hello,
I'm asking for arm64 architecture support. Is this possible ?
Hi there! Glad to see someone else is using this.
Honestly this hadn't crossed my mind. Is this something I'd be able to test with my raspberry pi?
In theory it should be possible:
https://launchpad.net/ubuntu/xenial/arm64/s3fs/1.79+git90-g8f11507-2
I'm not really a developer, so I don't know so much about cross-compiling. It might be as simple as substituting the amd64 .deb for the arm64. If you want to try and figure it out I'd be happy to get a pull request.
I won't have any time to look at this until this weekend at the very earliest.
Hey,
As compiling is always a pain in the a**, i'm always looking for alternative solution, glad to found yours :)
Raspberry pi is arm and not arm64 so i'm afraid you wont be able to do that from scratch.
It does not need crosscompiling, after reading your repo, I think it may works without any changes, just using docker buildx (multiarch) feature.
I started myself with this : https://mirailabs.io/blog/multiarch-docker-with-buildx/
Will try to build your dockerfile myself if I find some free time soon
Cool, thanks!!! Do you mind if I ask what your use case is? What sort of hardware do you have which runs arm64?
Some hosting solutions offers arm64 architecture, it is generally cheaper (because the hardware is cheaper), and I have several of thoses running at Scaleway, nothing fancy.
Almosted finished to build with docker buildx for arm64 btw
Sad,
the build failed almost at the end
=> ERROR [15/17] RUN 060-build-package.sh 596.7s
------
> [15/17] RUN 060-build-package.sh:
#20 0.257 ++ find . -maxdepth 1 -name 's3fs-fuse-*' -type d
#20 0.325 + PACKAGE_DIR=./s3fs-fuse-1.86+git-v1.86
#20 0.331 + cd ./s3fs-fuse-1.86+git-v1.86
#20 0.334 + sed -i s/libcurl4-gnutls-dev/libcurl4-openssl-dev/g debian/control
#20 0.424 + sed -i s/--with-gnutls/--with-openssl/g debian/rules
#20 0.498 + '[' '2020-02-16 18:00:00' '!=' NONE ']'
#20 0.508 ++ date -u +%s
#20 0.558 ++ date -u '--date=2020-02-16 18:00:00' +%s
#20 0.601 + '[' 1585590159 -lt 1581876000 ']'
#20 0.609 ++ date -u -R '--date=2020-02-16 18:00:00'
#20 0.654 + sed -i '0,/> / s/> .*/> Sun, 16 Feb 2020 18:00:00 +0000/g' debian/changelog
#20 0.726 + find . -exec touch -m -d '2020-02-16 18:00:00' '{}' +
#20 1.115 + debuild -b -uc -us
#20 4.360 dpkg-buildpackage -rfakeroot -us -uc -ui -b
#20 6.670 dpkg-buildpackage: info: source package s3fs-fuse
#20 6.670 dpkg-buildpackage: info: source version 1.86+git-v1.86-2
#20 6.670 dpkg-buildpackage: info: source distribution UNRELEASED
#20 6.670 dpkg-buildpackage: info: source changed by Ben Mares <services-docker-build-s3fs@tensorial.com>
#20 7.931 dpkg-source --before-build s3fs-fuse-1.86+git-v1.86
#20 7.931 dpkg-buildpackage: info: host architecture arm64
#20 12.15 fakeroot debian/rules clean
#20 13.67 dh clean
#20 15.52 dh_clean
#20 16.84 debian/rules build
#20 16.92 dh build
#20 18.52 dh_update_autotools_config
#20 19.72 dh_autoreconf
#20 64.36 configure.ac:30: installing './compile'
#20 64.40 configure.ac:26: installing './config.guess'
#20 64.46 configure.ac:26: installing './config.sub'
#20 64.51 configure.ac:27: installing './install-sh'
#20 64.56 configure.ac:27: installing './missing'
#20 65.61 src/Makefile.am: installing './depcomp'
#20 65.86 parallel-tests: installing './test-driver'
#20 67.02 debian/rules override_dh_auto_configure
#20 67.09 make[1]: Entering directory '/home/deb/s3fs-fuse-1.86+git-v1.86'
#20 67.09 dh_auto_configure -- --with-openssl
#20 68.22 ./configure --build=aarch64-linux-gnu --prefix=/usr --includedir=\${prefix}/include --mandir=\${prefix}/share/man --infodir=\${prefix}/share/info --sysconfdir=/etc --localstatedir=/var --disable-silent-rules --libdir=\${prefix}/lib/aarch64-linux-gnu --libexecdir=\${prefix}/lib/aarch64-linux-gnu --disable-maintainer-mode --disable-dependency-tracking --with-openssl
#20 72.24 configure: WARNING: unrecognized options: --disable-maintainer-mode
#20 73.93 checking build system type... aarch64-unknown-linux-gnu
#20 74.49 checking host system type... aarch64-unknown-linux-gnu
#20 74.50 checking target system type... aarch64-unknown-linux-gnu
#20 74.50 checking for a BSD-compatible install... /usr/bin/install -c
#20 74.82 checking whether build environment is sane... yes
#20 75.22 checking for a thread-safe mkdir -p... /bin/mkdir -p
#20 75.28 checking for gawk... no
#20 75.29 checking for mawk... mawk
#20 75.29 checking whether make sets $(MAKE)... yes
#20 75.70 checking whether make supports nested variables... yes
#20 76.05 checking for g++... g++
#20 76.84 checking whether the C++ compiler works... yes
#20 78.47 checking for C++ compiler default output file name... a.out
#20 78.51 checking for suffix of executables...
#20 80.15 checking whether we are cross compiling... no
#20 82.56 checking for suffix of object files... o
#20 83.69 checking whether we are using the GNU C++ compiler... yes
#20 84.72 checking whether g++ accepts -g... yes
#20 85.67 checking for style of include used by make... GNU
#20 85.85 checking dependency style of g++... none
#20 85.88 checking for gcc... gcc
#20 86.68 checking whether we are using the GNU C compiler... yes
#20 87.68 checking whether gcc accepts -g... yes
#20 88.54 checking for gcc option to accept ISO C89... none needed
#20 90.92 checking whether gcc understands -c and -o together... yes
#20 92.42 checking dependency style of gcc... none
#20 92.43 checking how to run the C preprocessor... gcc -E
#20 94.29 checking for grep that handles long lines and -e... /bin/grep
#20 94.36 checking for egrep... /bin/grep -E
#20 94.43 checking for ANSI C header files... yes
#20 99.35 checking for sys/types.h... yes
#20 100.5 checking for sys/stat.h... yes
#20 101.7 checking for stdlib.h... yes
#20 103.0 checking for string.h... yes
#20 104.3 checking for memory.h... yes
#20 105.6 checking for strings.h... yes
#20 107.1 checking for inttypes.h... yes
#20 108.3 checking for stdint.h... yes
#20 109.5 checking for unistd.h... yes
#20 110.7 checking sys/xattr.h usability... yes
#20 111.7 checking sys/xattr.h presence... yes
#20 112.0 checking for sys/xattr.h... yes
#20 112.1 checking attr/xattr.h usability... no
#20 113.2 checking attr/xattr.h presence... no
#20 113.7 checking for attr/xattr.h... no
#20 113.7 checking sys/extattr.h usability... no
#20 114.8 checking sys/extattr.h presence... no
#20 115.3 checking for sys/extattr.h... no
#20 115.3 checking s3fs build with nettle(GnuTLS)... no
#20 115.3 checking s3fs build with OpenSSL... yes
#20 115.3 checking s3fs build with GnuTLS... no
#20 115.3 checking s3fs build with NSS... no
#20 115.4 checking for pkg-config... /usr/bin/pkg-config
#20 115.4 checking pkg-config is at least version 0.9.0... yes
#20 115.4 checking for common_lib_checking... yes
#20 115.7 checking compile s3fs with... OpenSSL
#20 115.8 checking for DEPS... yes
#20 116.1 checking for malloc_trim... yes
#20 117.7 checking for library containing clock_gettime... none required
#20 119.2 checking for clock_gettime... yes
#20 120.8 checking pthread mutex recursive... PTHREAD_MUTEX_RECURSIVE
#20 121.9 checking checking CURLOPT_TCP_KEEPALIVE... yes
#20 123.3 checking checking CURLOPT_SSL_ENABLE_ALPN... yes
#20 124.6 checking checking CURLOPT_KEEP_SENDING_ON_ERROR... yes
#20 126.0 checking for git... yes
#20 126.0 checking github short commit hash... v1.86
#20 126.6 checking that generated files are newer than configure... done
#20 126.6 configure: creating ./config.status
#20 129.7 config.status: creating Makefile
#20 130.1 config.status: creating src/Makefile
#20 130.7 config.status: creating test/Makefile
#20 131.3 config.status: creating doc/Makefile
#20 131.9 config.status: creating config.h
#20 132.2 config.status: executing depfiles commands
#20 132.2 configure: WARNING: unrecognized options: --disable-maintainer-mode
#20 132.8 make[1]: Leaving directory '/home/deb/s3fs-fuse-1.86+git-v1.86'
#20 133.0 dh_auto_build
#20 133.9 make -j4 -O
#20 134.1 /usr/bin/make all-recursive
#20 134.5 Making all in src
#20 224.2 make[3]: Entering directory '/home/deb/s3fs-fuse-1.86+git-v1.86/src'
#20 224.2 g++ -DHAVE_CONFIG_H -I. -I.. -D_FILE_OFFSET_BITS=64 -I/usr/include/fuse -I/usr/include/aarch64-linux-gnu -I/usr/include/libxml2 -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/home/deb/s3fs-fuse-1.86+git-v1.86=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -c -o string_util.o string_util.cpp
#20 224.2 make[3]: Leaving directory '/home/deb/s3fs-fuse-1.86+git-v1.86/src'
#20 262.8 make[3]: Entering directory '/home/deb/s3fs-fuse-1.86+git-v1.86/src'
#20 262.8 g++ -DHAVE_CONFIG_H -I. -I.. -D_FILE_OFFSET_BITS=64 -I/usr/include/fuse -I/usr/include/aarch64-linux-gnu -I/usr/include/libxml2 -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/home/deb/s3fs-fuse-1.86+git-v1.86=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -c -o cache.o cache.cpp
#20 262.8 make[3]: Leaving directory '/home/deb/s3fs-fuse-1.86+git-v1.86/src'
#20 384.2 make[3]: Entering directory '/home/deb/s3fs-fuse-1.86+git-v1.86/src'
#20 384.2 g++ -DHAVE_CONFIG_H -I. -I.. -D_FILE_OFFSET_BITS=64 -I/usr/include/fuse -I/usr/include/aarch64-linux-gnu -I/usr/include/libxml2 -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/home/deb/s3fs-fuse-1.86+git-v1.86=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -c -o s3fs_util.o s3fs_util.cpp
#20 384.2 s3fs_util.cpp: In function 'std::__cxx11::string get_realpath(const char*)':
#20 384.2 s3fs_util.cpp:67:1: internal compiler error: Segmentation fault
#20 384.2 }
#20 384.2 ^
#20 384.2 mmap: Permission denied
#20 384.2 Please submit a full bug report,
#20 384.2 with preprocessed source if appropriate.
#20 384.2 See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
#20 384.2 The bug is not reproducible, so it is likely a hardware or OS problem.
#20 384.2 Makefile:619: recipe for target 's3fs_util.o' failed
#20 384.2 make[3]: *** [s3fs_util.o] Error 1
#20 384.2 make[3]: Leaving directory '/home/deb/s3fs-fuse-1.86+git-v1.86/src'
#20 384.2 make[3]: *** Waiting for unfinished jobs....
#20 516.1 make[3]: Entering directory '/home/deb/s3fs-fuse-1.86+git-v1.86/src'
#20 516.1 g++ -DHAVE_CONFIG_H -I. -I.. -D_FILE_OFFSET_BITS=64 -I/usr/include/fuse -I/usr/include/aarch64-linux-gnu -I/usr/include/libxml2 -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/home/deb/s3fs-fuse-1.86+git-v1.86=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -c -o s3fs.o s3fs.cpp
#20 516.1 make[3]: Leaving directory '/home/deb/s3fs-fuse-1.86+git-v1.86/src'
#20 517.9 make[3]: Entering directory '/home/deb/s3fs-fuse-1.86+git-v1.86/src'
#20 517.9 g++ -DHAVE_CONFIG_H -I. -I.. -D_FILE_OFFSET_BITS=64 -I/usr/include/fuse -I/usr/include/aarch64-linux-gnu -I/usr/include/libxml2 -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/home/deb/s3fs-fuse-1.86+git-v1.86=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -c -o fdcache.o fdcache.cpp
#20 517.9 make[3]: Leaving directory '/home/deb/s3fs-fuse-1.86+git-v1.86/src'
#20 596.5 make[3]: Entering directory '/home/deb/s3fs-fuse-1.86+git-v1.86/src'
#20 596.5 g++ -DHAVE_CONFIG_H -I. -I.. -D_FILE_OFFSET_BITS=64 -I/usr/include/fuse -I/usr/include/aarch64-linux-gnu -I/usr/include/libxml2 -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/home/deb/s3fs-fuse-1.86+git-v1.86=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -c -o curl.o curl.cpp
#20 596.5 curl.cpp: In function 'std::__cxx11::string url_to_host(const string&)':
#20 596.5 curl.cpp:120:1: internal compiler error: Segmentation fault
#20 596.5 }
#20 596.5 ^
#20 596.5 mmap: Permission denied
#20 596.5 Please submit a full bug report,
#20 596.5 with preprocessed source if appropriate.
#20 596.5 See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
#20 596.5 The bug is not reproducible, so it is likely a hardware or OS problem.
#20 596.5 Makefile:619: recipe for target 'curl.o' failed
#20 596.5 make[3]: *** [curl.o] Error 1
#20 596.5 make[3]: Leaving directory '/home/deb/s3fs-fuse-1.86+git-v1.86/src'
#20 596.5 Makefile:400: recipe for target 'all-recursive' failed
#20 596.5 make[2]: *** [all-recursive] Error 1
#20 596.5 Makefile:341: recipe for target 'all' failed
#20 596.5 make[1]: *** [all] Error 2
#20 596.6 dh_auto_build: make -j4 -O returned exit code 2
#20 596.6 debian/rules:6: recipe for target 'build' failed
#20 596.6 make: *** [build] Error 25
#20 596.6 dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2
#20 596.6 debuild: fatal error at line 1152:
#20 596.6 dpkg-buildpackage -rfakeroot -us -uc -ui -b failed
Haha, segfault in the compiler! I think that's an achievement. :)
A few thoughts:
- Have you checked to see if it's pulling the arm64
.debfile? - Will it compile the code from the .deb file with the original source code (without replacing it with the GitLab source code)?
I tried again, no segfault then. Guess that was my computer doing too much in the same time. Will try to get the deb and test it
EDIT: Can not find out it is saying that
docker cp $id:s3fs-debian-package.tar .
Error: No such container:path: 743f79292a0e28a6a6a7d045d3cc9f2abfbd58d4b27d87425eb31e4d3e28cafa:s3fs-debian-package.tar
EDIT2 : Silly me, that's just and example isnt it ?
there it is :
deb@dcdf946cd907:~$ ls
s3fs-dbgsym_1.86+git-v1.86-2_arm64.ddeb s3fs-fuse_1.82-1.dsc s3fs-fuse_1.86+git-v1.86-2_arm64.buildinfo
s3fs-fuse-1.86+git-v1.86 s3fs-fuse_1.82.orig.tar.gz s3fs-fuse_1.86+git-v1.86-2_arm64.changes
s3fs-fuse_1.82-1.debian.tar.xz s3fs-fuse_1.86+git-v1.86-2_arm64.build s3fs_1.86+git-v1.86-2_arm64.deb
EDIT3 : It works!
Example (from the link I previously added) :
export DOCKER_CLI_EXPERIMENTAL=enabled
docker run --rm --privileged docker/binfmt:66f9012c56a8316f9244ffd7622d7c21c1f6f28d
docker buildx create --use
docker buildx build -t test --platform=linux/aarch64,linux/amd64,linux/arm .
You can also add linux/aarch64 for raspberry pi :)
What about you testing?
Nice work!!! I'll try to test this weekend.
Do you think we could get this properly written up and added?
Also, do you know if there's any way around using --privileged? One of my goals is to keep this build process as safe and transparent as possible, and running that privileged command with the long hex code gives me the creeps.
Well, as such there is not much to add, the docker buildx is an alternative to not having to build directly on the arch you want, so a bit of readme eventually but that's all ?
About the docker run --rm --privileged docker/binfmt:66f9012c56a8316f9244ffd7622d7c21c1f6f28d
I have found this official docker image on several forums or good looking tutorials like the one I gave, but nothing official and it bothers me too, but this is unblocked the unavailable architectures on my laptop. But I do not need it when building with gitlab-ci on my virtual machines tho.
I suggest that you could add the tested architectures on your readme, and try docker buildx multiarch feature by yourself.
@NoxInmortus thanks for closing this, but I would like to address this issue at some point. I actually made an attempt at cross-compiling about two weeks ago, but I was not successful. (I just don't have enough experience to be able to properly diagnose things when a build fails.)
I also recently came across a multi-arch version of conda, and I wonder if I could adapt their pipeline. They use something called qemu-user-static, but I'd need to look into it further to understand how it works.
Hello @maresb, hope you're doing fine in those troubled times.
I very recently ended a short project to quickly build some debian package for many differents architectures. I've almost never built packages before, but I finally made it. Looking back on this issue I think it might interrest you if you need some references.
Here is the repository : https://github.com/NoxInmortus/docker-build-pqchecker
I'm using gitlab-runner and docker buildx to build the images.
I'm using this ansible role to install gitlab-runner : https://gitlab.com/guardianproject-ops/ansible-gitlab-runner.git
And this set of task to allow gitlab-runner to run correctly docker buildx :
---
- name: gitlab-runner | Mkdir ${HOME}/.docker/cli-plugins
file:
state: directory
path: /home/gitlab-runner/.docker/cli-plugins
owner: gitlab-runner
group: gitlab-runner
- name: gitlab-runner | rm ${HOME}/.bash_logout
file:
state: absent
path: /home/gitlab-runner/.bash_logout
- name: gitlab-runner | Get docker-pushrm cli-plugin
shell: 'wget -q https://github.com/christian-korneck/docker-pushrm/releases/download/v1.5.1/docker-pushrm_linux_amd64 -O /home/gitlab-runner/.docker/cli-plugins/docker-pushrm'
args:
creates: /home/gitlab-runner/.docker/cli-plugins/docker-pushrm
- name: gitlab-runner | Get docker-buildx cli-plugin
shell: 'wget -q https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-amd64 -O /home/gitlab-runner/.docker/cli-plugins/docker-buildx'
args:
creates: /home/gitlab-runner/.docker/cli-plugins/docker-buildx
- name: gitlab-runner | Chown/Chmod docker cli-plugins
file:
path: '/home/gitlab-runner/.docker/cli-plugins/docker-{{ item }}'
owner: gitlab-runner
group: gitlab-runner
mode: '0755'
with_items:
- pushrm
- buildx
- name: gitlab-runner | Add gitlab-runner to docker group
user:
name: gitlab-runner
groups:
- docker
notify: docker_restart
- name: gitlab-runner | Enable experimental docker features
copy:
dest: /etc/docker/config.json
content: |
{
"experimental": true
}
notify: docker_restart
- name: gitlab-runner | install qemu-user-static binfmt-support
apt:
name: '{{ item }}'
with_items:
- qemu-user-static
- binfmt-support
- name: gitlab-runner | docker buildx create --use --name multiarch
command: docker buildx create --use --name multiarch
become: yes
become_user: gitlab-runner
args:
creates: /home/gitlab-runner/.docker/buildx/instances/multiarch
Hope it helps.
Ooh, thanks so much for the info! I've had a lot of distractions come up over the past several weekends, but this is still on my mind. I will definitely have a look.