`tebako setup` failure on ubuntu arm64 container

[ronaldtse]

Running on Apple M1, using the arm64 ubuntu container (this is 20.04-lts).

$ docker run -it --platform linux/arm64 ubuntu bash

Within the container, run this to build tebako:

export DEBIAN_FRONTEND=noninteractive
export TZ=Etc/UTC

apt-get update
apt-get install -y curl git g++ gcc ruby ruby-dev pkg-config bison flex
curl https://apt.kitware.com/kitware-archive.sh | bash
apt-get install -y cmake; \
apt-get install -y binutils-dev libarchive-dev libevent-dev libjemalloc-dev acl-dev \
  libdouble-conversion-dev libiberty-dev liblz4-dev liblzma-dev libssl-dev \
  libboost-context-dev libboost-filesystem-dev libboost-program-options-dev \
  libboost-regex-dev libboost-system-dev libboost-thread-dev \
  libunwind-dev libdwarf-dev libelf-dev libfuse-dev libgoogle-glog-dev \
  libffi-dev libgdbm-dev libyaml-dev libncurses-dev libreadline-dev \
  libsqlite3-dev libfmt-dev; \
gem install ronn; \
gem install bundler

git clone https://github.com/metanorma/packed-mn.git && \
cd packed-mn && \
git checkout maxirmx-tebako-packager && \
git clone https://github.com/tamatebako/tebako && \
tebako/bin/tebako setup

It compiles and then fails here:

g++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
make[9]: *** [fbthrift/thrift/compiler/CMakeFiles/compiler_generators.dir/build.make:216: fbthrift/thrift/compiler/CMakeFiles/compiler_generators.dir/generate/t_mstch_cpp2_generator.cc.o] Error 1
make[9]: *** Waiting for unfinished jobs....
g++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
make[9]: *** [fbthrift/thrift/compiler/CMakeFiles/compiler_generators.dir/build.make:272: fbthrift/thrift/compiler/CMakeFiles/compiler_generators.dir/generate/t_mstch_py3_generator.cc.o] Error 1
g++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
make[9]: *** [fbthrift/thrift/compiler/CMakeFiles/compiler_generators.dir/build.make:132: fbthrift/thrift/compiler/CMakeFiles/compiler_generators.dir/generate/t_go_generator.cc.o] Error 1
make[8]: *** [CMakeFiles/Makefile2:1050: fbthrift/thrift/compiler/CMakeFiles/compiler_generators.dir/all] Error 2
make[7]: *** [Makefile:156: all] Error 2
make[6]: *** [CMakeFiles/_dwarfs.dir/build.make:86: /packed-mn/tebako/deps/src/_dwarfs_wr/deps/src/_dwarfs-stamp/_dwarfs-build] Error 2
make[5]: *** [CMakeFiles/Makefile2:87: CMakeFiles/_dwarfs.dir/all] Error 2
make[4]: *** [Makefile:136: all] Error 2
make[3]: *** [CMakeFiles/_dwarfs_wr.dir/build.make:86: ../deps/src/_dwarfs_wr-stamp/_dwarfs_wr-build] Error 2
make[2]: *** [CMakeFiles/Makefile2:114: CMakeFiles/_dwarfs_wr.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:175: CMakeFiles/setup.dir/rule] Error 2
make: *** [Makefile:163: setup] Error 2
'tebako setup' build step failed

[maxirmx]

It is internal compiler error. Probably it does not have enough memory.
Facebook now does not say at all how much is required to compile folly. Last time I saw it back in May, it was 8 Gb RAM requirement

[ronaldtse]

I updated Docker to run with 12GB RAM and tebako setup successfully finished.

In order to build packed-mn, I needed to do this:

Building packed-mn:

apt-get install -y libpng-dev libc6-dev libfontconfig1-dev libfreetype6-dev zlib1g-dev
bin/build-with-tebako.sh

Then I encounter this issue:

    current directory: /packed-mn/tebako/output/source_filesystem/lib/ruby/gems/2.7.0/gems/rice-3.0.0
/packed-mn/tebako/output/source_filesystem/bin/ruby -I /packed-mn/tebako/output/source_filesystem/lib/ruby/2.7.0 -r
./siteconf20211217-97085-wmcyqu.rb extconf.rb
Unfortunately Rice does not build against a staticly linked Ruby.
You'll need to rebuild Ruby with --enable-shared to use this library.

If you're on rvm:    rvm reinstall [version] -- --enable-shared
If you're on rbenv:  CONFIGURE_OPTS="--enable-shared" rbenv install [version]

How did you get around this?

[maxirmx]

@ronaldtse
The whole idea was to have --disable-shared. Otherwise Ruby builds extensions as shared libraries and requires all these libraries to run.

I did not have this issue, so it is something new for me.

[maxirmx]

Building packed-mn:

apt-get install -y libpng-dev libc6-dev libfontconfig1-dev libfreetype6-dev zlib1g-dev
bin/build-with-tebako.sh

I do not think it is related to tebako. packed-mn probably does have its own requirements that need to be satisfied.

[maxirmx]

1. rice gem was not a part of packed-mn package, so I did not see this issue
2. rice 3.0 patches ruby setup script. It looks like rice patch is not compatible with tebako patch. I am not sure I can fix it since rice is patching "in the opposite direction", to facilitate dynamic linking, while tebako is patching to facilitate static linking.
3. rice 4.0 uses conventional configuration facilities. I guess it will work with tebako.

There are two non-exclusive options:

• patch rice 3.0 patch so that it is compatible with tebako patch. Very shaky but may be possible
• just test rice 4.0

[ronaldtse]

just test rice 4.0

Good point, yes let me try that.

[ronaldtse]

To build packed-mn, I had to do this:

apt-get install -y libxslt-dev libz-dev libpng-dev libfreetype-dev libfontconfig-dev autoconf

# This is needed to build the expressir gem
export CC=clang; export CC=clang++
bundle

# Then some gems will fail to compile, such as nokogiri
export CC=gcc; export CXX=g++
gem install nokogiri -- --use-system-libraries

# run bundle again
bundle

I'm facing this now:

root@6db1a805a582:/packed-mn# make
mkdir -p build/bin/;
.archive/tebako/bin/tebako press -r "build/package" -e "metanorma" -o "build/bin/metanorma-linux-aarch64";
Setting up Linux environment
make: *** [Makefile:106: build/bin/metanorma-linux-aarch64] Error 1
root@6db1a805a582:/packed-mn# make clean
rm -rf build
root@6db1a805a582:/packed-mn# make
mkdir -p build/package/;
cp bin/metanorma build/package/metanorma
mkdir -p build/package/;
cp Gemfile build/package/Gemfile
mkdir -p build/package/;
cp Gemfile.lock build/package/Gemfile.lock
mkdir -p build/package/;
cp vendor/cacert.pem.mozilla build/package/cacert.pem.mozilla
mkdir -p build/package/;
cp -R vendor build/package/vendor
touch build/.package-ready
mkdir -p build/bin/;
.archive/tebako/bin/tebako press -r "build/package" -e "metanorma" -o "build/bin/metanorma-linux-aarch64";
Setting up Linux environment
make: *** [Makefile:106: build/bin/metanorma-linux-aarch64] Error 1

[ronaldtse]

My bad... retrying.

[ronaldtse]

This also worked with the same steps here: #56 (comment)

Going to update the README.

[ronaldtse]

Addressed in faea15d.