Bad https://docs.leaningtech.com/cheerp/Linux-build-instructions

Question

Bad https://docs.leaningtech.com/cheerp/Linux-build-instructions

mingodad opened this issue 2 years ago · 51 comments

When trying to build cheerp-3.0 on a Ubuntu 18.04 64bits I'm getting an error when trying to build cheerp-libs/webgles/webgles.cpp that says it can't find #include <string> and looking at https://launchpad.net/~leaningtech-dev/+archive/ubuntu/cheerp-ppa I can see that there is cheerp-libcxx and cheerp-libcxx-libcxxabi but in the instructions to build from source there is only -DLLVM_ENABLE_PROJECTS=clang and probably it should be -DLLVM_ENABLE_PROJECTS=clang,libcxx,libcxxabi to also build and install the correspond libcxx to be able to resolve #include <string> .

Answer 1 · 2023-06-13T14:35:01.000Z

Please make sure to follow all the build steps, libcxx and libcxxabi are built after musl here:

https://docs.leaningtech.com/cheerp/Linux-build-instructions#build-cheerp2-utilities-and-libraries-stable-version

Answer 2 · 2023-06-13T14:42:00.000Z

I don't see any mention to libcxx/libcxxabi there.

Are you sure about it ?

Here is my shell script adapted from that page to install on a local folder:

#!/bin/sh
MYCHEERP_PRFIX=$HOME/local/cheerp
#mkdir cheerp
#cd cheerp
export CHEERP_SRC=$PWD
git clone --depth 1 --branch cheerp-3.0 https://github.com/leaningtech/cheerp-compiler
git clone --depth 1 --branch cheerp-3.0 https://github.com/leaningtech/cheerp-utils
git clone --depth 1 --branch cheerp-3.0 https://github.com/leaningtech/cheerp-musl
git clone --depth 1 --branch cheerp-3.0 https://github.com/leaningtech/cheerp-libs

cd cheerp-compiler
sed -i '13a\  llvm-ar' llvm/CheerpCmakeConf.cmake
mycmake -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PRFIX -S llvm -B build -C llvm/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS=all -G Ninja
ninja -C build -j4
ninja -C build install
cd ..

cd cheerp-utils
mycmake -B build -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PRFIX .
make -C build install
cd ..

cd cheerp-musl
mkdir build_genericjs
cd build_genericjs
RANLIB="$MYCHEERP_PRFIX/bin/llvm-ar s" AR="$MYCHEERP_PRFIX/bin/llvm-ar"  CC="$MYCHEERP_PRFIX/bin/clang -target cheerp" LD="$MYCHEERP_PRFIX/bin/llvm-link" CFLAGS="-Wno-int-conversion" ../configure --target=cheerp --disable-shared --prefix=$MYCHEERP_PRFIX
make clean
make -j8
make install
cd ..
mkdir build_asmjs
cd build_asmjs
RANLIB="$MYCHEERP_PRFIX/bin/llvm-ar s" AR="$MYCHEERP_PRFIX/bin/llvm-ar"  CC="$MYCHEERP_PRFIX/bin/clang -target cheerp-wasm" LD="$MYCHEERP_PRFIX/bin/llvm-link" CFLAGS="-Wno-int-conversion" ../configure --target=cheerp-wasm --disable-shared --prefix=$MYCHEERP_PRFIX
make clean
make -j8
make install
cd ../..

cd cheerp-compiler
mycmake -S runtimes -B build_runtimes_genericjs -GNinja -C runtimes/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="$MYCHEERP_PRFIX/share/cmake/Modules/CheerpToolchain.cmake"
ninja -C build_runtimes_genericjs
ninja -C build_runtimes_genericjs install

mycmake -S runtimes -B build_runtimes_wasm -GNinja -C runtimes/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="$MYCHEERP_PRFIX/share/cmake/Modules/CheerpWasmToolchain.cmake"
ninja -C build_runtimes_wasm
ninja -C build_runtimes_wasm install
cd ..

cd cheerp-libs
make -C webgles install INSTALL_PREFIX=$MYCHEERP_PRFIX CHEERP_PREFIX=$MYCHEERP_PRFIX
make -C wasm install INSTALL_PREFIX=$MYCHEERP_PRFIX CHEERP_PREFIX=$MYCHEERP_PRFIX
make -C stdlibs install INSTALL_PREFIX=$MYCHEERP_PRFIX CHEERP_PREFIX=$MYCHEERP_PRFIX
cd ..

Answer 3 · 2023-06-13T14:45:33.000Z

The libraries are built as part of the runtime steps

Answer 4 · 2023-06-13T14:49:05.000Z

As you can see my copy/paste of the instructions and it fails:

/home/mingo/local/cheerp/bin/llvm-ar s lib/libc.a
../tools/install.sh -D -m 644 lib/Scrt1.o /home/mingo/local/cheerp/lib/asmjs/Scrt1.o
../tools/install.sh -D -m 644 lib/crt1.o /home/mingo/local/cheerp/lib/asmjs/crt1.o
../tools/install.sh -D -m 644 lib/crti.o /home/mingo/local/cheerp/lib/asmjs/crti.o
../tools/install.sh -D -m 644 lib/crtn.o /home/mingo/local/cheerp/lib/asmjs/crtn.o
../tools/install.sh -D -m 644 lib/rcrt1.o /home/mingo/local/cheerp/lib/asmjs/rcrt1.o
../tools/install.sh -D -m 644 lib/libc.a /home/mingo/local/cheerp/lib/asmjs/libc.a
../tools/install.sh -D -m 644 lib/libm.a /home/mingo/local/cheerp/lib/asmjs/libm.a
../tools/install.sh -D -m 644 lib/librt.a /home/mingo/local/cheerp/lib/asmjs/librt.a
../tools/install.sh -D -m 644 lib/libpthread.a /home/mingo/local/cheerp/lib/asmjs/libpthread.a
../tools/install.sh -D -m 644 lib/libcrypt.a /home/mingo/local/cheerp/lib/asmjs/libcrypt.a
../tools/install.sh -D -m 644 lib/libutil.a /home/mingo/local/cheerp/lib/asmjs/libutil.a
../tools/install.sh -D -m 644 lib/libxnet.a /home/mingo/local/cheerp/lib/asmjs/libxnet.a
../tools/install.sh -D -m 644 lib/libresolv.a /home/mingo/local/cheerp/lib/asmjs/libresolv.a
../tools/install.sh -D -m 644 lib/libdl.a /home/mingo/local/cheerp/lib/asmjs/libdl.a
../tools/install.sh -D -m 644 lib/libc.bc /home/mingo/local/cheerp/lib/asmjs/libc.bc
../tools/install.sh -D -m 644 lib/crt1.bc /home/mingo/local/cheerp/lib/asmjs/crt1.bc
../tools/install.sh -D -m 644 obj/include/bits/alltypes.h /home/mingo/local/cheerp/include/bits/alltypes.h
../tools/install.sh -D -m 644 obj/include/bits/syscall.h /home/mingo/local/cheerp/include/bits/syscall.h
loading initial cache file runtimes/CheerpCmakeConf.cmake
-- The ASM compiler identification is unknown
-- Didn't find assembler
CMake Error at CMakeLists.txt:3 (project):
  The CMAKE_C_COMPILER:

    /opt/cheerp/bin/clang

  is not a full path to an existing compiler tool.

  Tell CMake where to find the compiler by setting either the environment
  variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to
  the compiler, or to the compiler name if it is in the PATH.


CMake Error at CMakeLists.txt:3 (project):
  The CMAKE_CXX_COMPILER:

    /opt/cheerp/bin/clang++

  is not a full path to an existing compiler tool.

  Tell CMake where to find the compiler by setting either the environment
  variable "CXX" or the CMake cache entry CMAKE_CXX_COMPILER to the full path
  to the compiler, or to the compiler name if it is in the PATH.


CMake Error at CMakeLists.txt:3 (project):
  No CMAKE_ASM_COMPILER could be found.

  Tell CMake where to find the compiler by setting either the environment
  variable "ASM" or the CMake cache entry CMAKE_ASM_COMPILER to the full path
  to the compiler, or to the compiler name if it is in the PATH.


-- Warning: Did not find file Compiler/-ASM
-- Configuring incomplete, errors occurred!
See also "/tmp/cheerp/cheerp-compiler/build_runtimes_genericjs/CMakeFiles/CMakeOutput.log".
See also "/tmp/cheerp/cheerp-compiler/build_runtimes_genericjs/CMakeFiles/CMakeError.log".
ninja: Entering directory `build_runtimes_genericjs'
ninja: error: loading 'build.ninja': No such file or directory
ninja: Entering directory `build_runtimes_genericjs'
ninja: error: loading 'build.ninja': No such file or directory
loading initial cache file runtimes/CheerpCmakeConf.cmake
-- The ASM compiler identification is unknown
-- Didn't find assembler
CMake Error at CMakeLists.txt:3 (project):
  The CMAKE_C_COMPILER:

    /opt/cheerp/bin/clang

  is not a full path to an existing compiler tool.

  Tell CMake where to find the compiler by setting either the environment
  variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to
  the compiler, or to the compiler name if it is in the PATH.


CMake Error at CMakeLists.txt:3 (project):
  The CMAKE_CXX_COMPILER:

    /opt/cheerp/bin/clang++

  is not a full path to an existing compiler tool.

  Tell CMake where to find the compiler by setting either the environment
  variable "CXX" or the CMake cache entry CMAKE_CXX_COMPILER to the full path
  to the compiler, or to the compiler name if it is in the PATH.


CMake Error at CMakeLists.txt:3 (project):
  No CMAKE_ASM_COMPILER could be found.

  Tell CMake where to find the compiler by setting either the environment
  variable "ASM" or the CMake cache entry CMAKE_ASM_COMPILER to the full path
  to the compiler, or to the compiler name if it is in the PATH.


-- Warning: Did not find file Compiler/-ASM
-- Configuring incomplete, errors occurred!
See also "/tmp/cheerp/cheerp-compiler/build_runtimes_wasm/CMakeFiles/CMakeOutput.log".
See also "/tmp/cheerp/cheerp-compiler/build_runtimes_wasm/CMakeFiles/CMakeError.log".
ninja: Entering directory `build_runtimes_wasm'
ninja: error: loading 'build.ninja': No such file or directory
ninja: Entering directory `build_runtimes_wasm'
ninja: error: loading 'build.ninja': No such file or directory
make: Entering directory '/tmp/cheerp/cheerp-libs/webgles'
mkdir -p /home/mingo/local/cheerp/include/GLES2/
cp -v webgles.h gl2.h /home/mingo/local/cheerp/include/GLES2/
'webgles.h' -> '/home/mingo/local/cheerp/include/GLES2/webgles.h'
'gl2.h' -> '/home/mingo/local/cheerp/include/GLES2/gl2.h'
/home/mingo/local/cheerp/bin/clang++  -I/home/mingo/local/clang-15/include -I/home/mingo/local/include -std=c++1y -c webgles.cpp -o webgles.bc -target cheerp -O3
In file included from webgles.cpp:12:
In file included from ./webgles.h:15:
In file included from /home/mingo/local/cheerp/bin/../include/client/cheerp/client.h:15:
/home/mingo/local/cheerp/bin/../include/client/cheerp/types.h:16:10: fatal error: 'string' file not found
#include <string>
         ^~~~~~~~
1 error generated.
Makefile:12: recipe for target 'webgles.bc' failed
make: *** [webgles.bc] Error 1
make: Leaving directory '/tmp/cheerp/cheerp-libs/webgles'
make: Entering directory '/tmp/cheerp/cheerp-libs/wasm'
mkdir -p /home/mingo/local/cheerp/lib/
cp -v libwasm.bc /home/mingo/local/cheerp/lib/
'libwasm.bc' -> '/home/mingo/local/cheerp/lib/libwasm.bc'
make: Leaving directory '/tmp/cheerp/cheerp-libs/wasm'
make: Entering directory '/tmp/cheerp/cheerp-libs/stdlibs'
make: *** No rule to make target '/home/mingo/local/cheerp/lib/genericjs/libc++.bc', needed by 'genericjs/libstdlibs.full.bc'.  Stop.
make: Leaving directory '/tmp/cheerp/cheerp-libs/stdlibs'

Answer 5 · 2023-06-13T14:52:26.000Z

CMake Error at CMakeLists.txt:3 (project):
  The CMAKE_C_COMPILER:

    /opt/cheerp/bin/clang

Answer 6 · 2023-06-13T14:52:48.000Z

It seems that there is /opt/cheerp/* hardcoded in several places and that prevents install on a user defined location.
Would be nice to have all of then replaced by a macro/variable.

Answer 7 · 2023-06-13T14:55:41.000Z

Passing an explicit -DCMAKE_INSTALL_PREFIX=... to the runtimes build step should work.

It is much easier for us to debug problems if users have the same layout as we do though.

Answer 8 · 2023-06-13T15:01:33.000Z

Passing an explicit -DCMAKE_INSTALL_PREFIX=... to the runtimes build step should work.

It is much easier for us to debug problems if users have the same layout as we do though.

I'm trying it now.
Can that be added to the instructions ?
Because some calls to cmake have it and some don't.

Here is the main offenders:

/tmp/cheerp/cheerp-compiler/build_runtimes_genericjs/CMakeCache.txt:20:CHEERP_PREFIX:STRING=/opt/cheerp
/tmp/cheerp/cheerp-compiler/build_runtimes_genericjs/CMakeCache.txt:67:CMAKE_CXX_COMPILER:STRING=/opt/cheerp/bin/clang++
/tmp/cheerp/cheerp-compiler/build_runtimes_genericjs/CMakeCache.txt:90:CMAKE_C_COMPILER:STRING=/opt/cheerp/bin/clang
/tmp/cheerp/cheerp-compiler/build_runtimes_genericjs/CMakeCache.txt:137:CMAKE_INSTALL_PREFIX:STRING=/opt/cheerp
/tmp/cheerp/cheerp-compiler/build_runtimes_genericjs/CMakeCache.txt:139:CMAKE_LINKER:STRING=/opt/cheerp/bin/llvm-link
/tmp/cheerp/cheerp-compiler/build_runtimes_genericjs/CMakeCache.txt:164:CMAKE_MODULE_PATH:STRING=/opt/cheerp/share/cmake/Modules
/tmp/cheerp/cheerp-compiler/build_runtimes_wasm/CMakeCache.txt:20:CHEERP_PREFIX:STRING=/opt/cheerp
/tmp/cheerp/cheerp-compiler/build_runtimes_wasm/CMakeCache.txt:67:CMAKE_CXX_COMPILER:STRING=/opt/cheerp/bin/clang++
/tmp/cheerp/cheerp-compiler/build_runtimes_wasm/CMakeCache.txt:90:CMAKE_C_COMPILER:STRING=/opt/cheerp/bin/clang
/tmp/cheerp/cheerp-compiler/build_runtimes_wasm/CMakeCache.txt:137:CMAKE_INSTALL_PREFIX:STRING=/opt/cheerp
/tmp/cheerp/cheerp-compiler/build_runtimes_wasm/CMakeCache.txt:139:CMAKE_LINKER:STRING=/opt/cheerp/bin/llvm-link
/tmp/cheerp/cheerp-compiler/build_runtimes_wasm/CMakeCache.txt:164:CMAKE_MODULE_PATH:STRING=/opt/cheerp/share/cmake/Modules

Answer 9 · 2023-06-13T15:03:53.000Z

I am happy to add that to the instruction if you can confirm that it works :-)

CMAKE_INSTALL_PREFIX should be enough, other definitions are derived from it

Answer 10 · 2023-06-13T15:05:27.000Z

Thanks !
It'll take around an hour to finish the build and if everything is OK I'll tell you.

Answer 11 · 2023-06-13T15:18:37.000Z

Also the number of threads to use are hardcoded and they don't match:

cd cheerp-compiler
...
ninja -C build -j4
...
cd cheerp-musl
...
make -j8
...
cd build_asmjs
...
make -j8

Probably removing the -j4 from ninja -C build -j4 and using make -j `nproc` .

Answer 12 · 2023-06-13T18:06:53.000Z

It doesn't build !

I've tried to build with g++-9.4 and with clang-15, none of then build, I'll look in more detail later.

Bellow I'm attaching the output of the build.

build.log.zip

Answer 13 · 2023-06-13T18:12:13.000Z

The problem is that both the genericjs and the wasm version of the runtimes must be compiled before any of them are installed.

I have fixed README.md file to reflect this:

https://github.com/leaningtech/cheerp-compiler/blob/master/README.md#build-libcxx-and-libcxxabi

Answer 14 · 2023-06-13T18:27:55.000Z

And what about the info here https://docs.leaningtech.com/cheerp/Linux-build-instructions ?

Answer 15 · 2023-06-13T18:35:37.000Z

Please follow the README, we will update the instructions on the website later

Answer 16 · 2023-06-14T07:57:51.000Z

Here is again the output of my attempt to build cheerp after changing the order pointed here #147 (comment) .

cheerp-failed-build.log.zip

Answer 17 · 2023-06-14T08:12:12.000Z

Make sure to wipe out the installation directory and try again.

Answer 18 · 2023-06-14T08:19:25.000Z

Can you try the script attached that I'm using or create your own based on the build instructions and try build on fresh machine/vm ?
Or even better can a github action be made to test it ?

Answer 19 · 2023-06-14T08:26:23.000Z

I also noticed that cheerp seems to be limited to 32bits:

cheerp-latest/bin/clang++ -x c /dev/null -dM -E

#define _ILP32 1
#define ASMJS 1
#define __ATOMIC_ACQUIRE 2
...
#define SIZEOF_LONG 4
#define SIZEOF_POINTER 4
#define SIZEOF_PTRDIFF_T 4
#define SIZEOF_SHORT 2
#define SIZEOF_SIZE_T 4
#define SIZEOF_WCHAR_T 4
...
#define SIZE_MAX 4294967295U
#define SIZE_TYPE unsigned int
#define SIZE_WIDTH 32

Answer 20 · 2023-06-14T08:29:58.000Z

We build every commit on CI and PPA, beside the order of installation corrected above they will work.

The installation directly must be clean, this is of course the case when building on CI.

The Web platform is considered to be 32-bit. There is some work on 64-bit wasm but it's not mature and will have significant performance issues.

Answer 21 · 2023-06-14T09:07:30.000Z

Why on #147 (comment) it indicates to build https://github.com/leaningtech/cheerp-newlib (that is archived) but in https://docs.leaningtech.com/cheerp/Linux-build-instructions it indicated to build https://github.com/leaningtech/cheerp-musl ?

Which one is the one ?

Answer 22 · 2023-06-14T09:16:49.000Z

Only musl, we need to review the docs, but the runtime steps are correct.

Answer 23 · 2023-06-14T09:29:45.000Z

I was looking at https://github.com/leaningtech/cheerp-compiler/blob/master/.circleci/config.yml and could see that you only build with debian/rpm but not the naked build described in https://docs.leaningtech.com/cheerp/Linux-build-instructions.

Can the naked build described in https://docs.leaningtech.com/cheerp/Linux-build-instructions be added to the circleci to allways check that it's working ?

Answer 24 · 2023-06-14T09:31:23.000Z

We consider the Debian build to be our reference build.

The problems you are observing are due to the README being out-of-sync with the Debian instruction. Having another copy of the instructions would not help.

Answer 25 · 2023-06-14T09:32:57.000Z

It does help because in doubt I could see a working recipe on https://github.com/leaningtech/cheerp-compiler/blob/master/.circleci/config.yml .

Answer 26 · 2023-06-14T09:35:20.000Z

You can see the recipe in the debian packages. It is equivalent.

I expect the build will work if you start with a fresh build directory, using musl and fixing the build order for the runtimes. Please do so, after you confirm the build work I will update the docs

Answer 27 · 2023-06-14T09:36:35.000Z

That was what I did and the result is there https://github.com/leaningtech/cheerp-meta/files/11743172/cheerp-failed-build.log.zip .

Answer 28 · 2023-06-14T09:41:08.000Z

Again, you need to completely remove the installation directory before.

Answer 29 · 2023-06-14T09:41:54.000Z

And that was what I did !

Answer 30 · 2023-06-14T09:46:37.000Z

I can see from the log that it is not the case.

In this log the runtimes fails at the genericjs step, which is the first one. This happens because a version is already installed in the destination directory, hence you have not removed the destination directory as requested.

In the original log the runtimes failed at the wasm step, which is the second one. This happened because the genericjs version was installed just before.

The build system of libc++ is broken (in upstream LLVM we believe) and cannot deal with the libraries being already installed in the system.

The solution to this problem is to install both the genericjs and wasm versions after both are built. This can only work in a fresh installation directory.

Answer 31 · 2023-06-14T10:00:16.000Z

Obs.: edited to remove my bad interpretation of the cmake debian recipe.
Obs.: edited again with an cat $0 to log the entire build proccess and restarted it again.

I'm redoing it now with rm -R * on top of my script:

#!/bin/sh
cat $0
export PATH=$HOME/local/clang-16/bin:$PATH
export CC=clang
export CXX=clang++

MYCHEERP_PREFIX=$HOME/local/cheerp-latest
#mkdir cheerp
#cd cheerp
export CHEERP_SRC=$PWD

rm -R $MYCHEERP_PREFIX
rm -R cheerp-compiler
rm -R cheerp-utils
rm -R cheerp-musl
rm -R cheerp-libs


git clone --depth 1  https://github.com/leaningtech/cheerp-compiler
git clone --depth 1  https://github.com/leaningtech/cheerp-utils
git clone --depth 1  https://github.com/leaningtech/cheerp-musl
git clone --depth 1  https://github.com/leaningtech/cheerp-libs

cd cheerp-compiler
mycmake -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -S llvm -B build -C llvm/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS=clang -G Ninja
ninja -C build
ninja -C build install
cd ..

cd cheerp-utils
mycmake -B build -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX .
make -C build install
cd ..

cd cheerp-musl
mkdir build_genericjs
cd build_genericjs
RANLIB="$MYCHEERP_PREFIX/bin/llvm-ar s" AR="$MYCHEERP_PREFIX/bin/llvm-ar"  CC="$MYCHEERP_PREFIX/bin/clang -target cheerp" LD="$MYCHEERP_PREFIX/bin/llvm-link" CFLAGS="-Wno-int-conversion" ../configure --target=cheerp --disable-shared --prefix=$MYCHEERP_PREFIX
make clean
make -j `nproc`
make install
cd ..
mkdir build_asmjs
cd build_asmjs
RANLIB="$MYCHEERP_PREFIX/bin/llvm-ar s" AR="$MYCHEERP_PREFIX/bin/llvm-ar"  CC="$MYCHEERP_PREFIX/bin/clang -target cheerp-wasm" LD="$MYCHEERP_PREFIX/bin/llvm-link" CFLAGS="-Wno-int-conversion" ../configure --target=cheerp-wasm --disable-shared --prefix=$MYCHEERP_PREFIX
make clean
make -j `nproc`
make install
cd ../..

cd cheerp-compiler
mycmake  -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -S runtimes -B build_runtimes_genericjs -GNinja -C runtimes/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="$MYCHEERP_PREFIX/share/cmake/Modules/CheerpToolchain.cmake"
ninja -C build_runtimes_genericjs

mycmake  -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -S runtimes -B build_runtimes_wasm -GNinja -C runtimes/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="$MYCHEERP_PREFIX/share/cmake/Modules/CheerpWasmToolchain.cmake"
ninja -C build_runtimes_wasm

ninja -C build_runtimes_genericjs install
ninja -C build_runtimes_wasm install
cd ..

cd cheerp-libs
make -C webgles install INSTALL_PREFIX=$MYCHEERP_PREFIX CHEERP_PREFIX=$MYCHEERP_PREFIX
make -C wasm install INSTALL_PREFIX=$MYCHEERP_PREFIX CHEERP_PREFIX=$MYCHEERP_PREFIX
make -C stdlibs install INSTALL_PREFIX=$MYCHEERP_PREFIX CHEERP_PREFIX=$MYCHEERP_PREFIX
cd ..

Answer 32 · 2023-06-14T10:10:44.000Z

I'm redoing it now with rm -R * on top of my script:

Just to clarify, it's the installation directory that needs to be cleaned. Not the build directory.

Answer 33 · 2023-06-14T10:13:02.000Z

Just to not have any doubt in future conversations after I publish the build.log !

Answer 34 · 2023-06-14T11:17:43.000Z

Somehow it seems I've been missing cleaning out before build, see bellow the output of:

/usr/bin/time sh mk-it-last.sh > build.log 2>&1

buildcheerp-latest.log.zip

But now with the fresh cheerp-latest build and trying the hello.cpp I'm getting this error:

// The cheerp/clientlib.h header contains declarations for the browser APIs
#include <cheerp/clientlib.h>

// webMain is the entry point for web applications written in Cheerp
void webMain()
{
        client::console.log("Hello, World Wide Web!");
}

Running:

cheerp-env clang++ -target cheerp hello.cpp -o hello.js
/home/mingo/local/cheerp-latest/bin/llvm-link: No such file or directory
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)

Answer 35 · 2023-06-14T11:19:08.000Z

Output with -v:

cheerp-env clang++ -v -target cheerp hello.cpp -o hello.js
clang version 16.0.0 (https://github.com/leaningtech/cheerp-compiler 528d56485decf17df06231e136bafb149eac300f)
Target: cheerp-leaningtech-webbrowser-genericjs
Thread model: posix
InstalledDir: /home/mingo/local/cheerp-latest/bin
 "/home/mingo/local/cheerp-latest/bin/clang-16" -cc1 -triple cheerp-leaningtech-webbrowser-genericjs -emit-llvm-bc -emit-llvm-uselists -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name hello.cpp -mrelocation-model static -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -fno-verbose-asm -no-integrated-as -mconstructor-aliases -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=gdb -fno-dwarf-directory-asm -v -fcoverage-compilation-dir=/home/mingo/dev/cheerp -resource-dir /home/mingo/local/cheerp-latest/lib/clang/16 -internal-isystem /home/mingo/local/cheerp-latest/bin/../include/c++/v1 -internal-isystem /home/mingo/local/cheerp-latest/lib/clang/16/include -internal-externc-isystem /home/mingo/local/cheerp-latest/bin/../include -internal-externc-isystem /home/mingo/local/cheerp-latest/bin/../include/client -no-opaque-pointers -fdeprecated-macro -fdebug-compilation-dir=/home/mingo/dev/cheerp -ferror-limit 19 -fno-rtti -fgnuc-version=4.2.1 -fno-threadsafe-statics -fcolor-diagnostics -o /tmp/hello-03b807.bc -x c++ hello.cpp
clang -cc1 version 16.0.0 based upon LLVM 16.0.0git default target cheerp-leaningtech-webbrowser-wasm
#include "..." search starts here:
#include <...> search starts here:
 /home/mingo/local/cheerp-latest/bin/../include/c++/v1
 /home/mingo/local/cheerp-latest/lib/clang/16/include
 /home/mingo/local/cheerp-latest/bin/../include
 /home/mingo/local/cheerp-latest/bin/../include/client
End of search list.
 "/home/mingo/local/cheerp-latest/bin/llvm-link" -o /tmp/hello-91c042.bc /tmp/hello-03b807.bc /home/mingo/local/cheerp-latest/bin/../lib/genericjs/libstdlibs.bc libsystem.bc
/home/mingo/local/cheerp-latest/bin/llvm-link: No such file or directory
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)

Answer 36 · 2023-06-14T11:25:15.000Z

"/home/mingo/local/cheerp-latest/bin/llvm-link" -o /tmp/hello-91c042.bc /tmp/hello-03b807.bc /home/mingo/local/cheerp-latest/bin/../lib/genericjs/libstdlibs.bc libsystem.bc

You are missing the libsystem.bc file, there is an additional step missing from the build doc. Please see the cheerp-libs README instead

https://github.com/leaningtech/cheerp-libs/#cheerp-syscall-library-installation-only-generic-js-version

Answer 37 · 2023-06-14T11:39:25.000Z

After adding this to my build script:

cd system
$cmake_cmd -B build_genericjs -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -DCMAKE_TOOLCHAIN_FILE=$MYCHEERP_PREFIX/share/cmake/Modules/CheerpToolchain.cmake -DCMAKE_BUILD_TYPE=Release .
make -C build_genericjs
make install -C build_genericjs
cd ..

It now build the hello.cpp.

But I also see that there are a lot more than in the build script:

make -C webgles install INSTALL_PREFIX=/opt/cheerp CHEERP_PREFIX=/opt/cheerp
make -C wasm install INSTALL_PREFIX=/opt/cheerp CHEERP_PREFIX=/opt/cheerp
make -C stdlibs install INSTALL_PREFIX=/opt/cheerp CHEERP_PREFIX=/opt/cheerp
make -C stdlibs install_genericjs INSTALL_PREFIX=/opt/cheerp CHEERP_PREFIX=/opt/cheerp
make -C stdlibs install_asmjs INSTALL_PREFIX=/opt/cheerp CHEERP_PREFIX=/opt/cheerp
cd system
cmake -B build_genericjs -DCMAKE_INSTALL_PREFIX=/opt/cheerp -DCMAKE_TOOLCHAIN_FILE=/opt/cheerp/share/cmake/Modules/CheerpToolchain.cmake -DCMAKE_BUILD_TYPE=Release .
make -C build_genericjs
make install -C build_genericjs
cd ..
cmake -B build_asmjs -DCMAKE_INSTALL_PREFIX=/opt/cheerp -DCMAKE_TOOLCHAIN_FILE=/opt/cheerp/share/cmake/Modules/CheerpWasmToolchain.cmake -DCMAKE_BUILD_TYPE=Release .
make -C build_asmjs
make install -C build_asmjs
cd ..

Somehow they need be in the build script ?

And here is the full build script:

#!/bin/sh
cat $0
export PATH=$HOME/local/clang-16/bin:$PATH
export CC=clang
export CXX=clang++
cmake_cmd=mycmake
rmdir_cmd="rm -R"

MYCHEERP_PREFIX=$HOME/local/cheerp-latest
#mkdir cheerp
#cd cheerp
export CHEERP_SRC=$PWD

$rmdir_cmd $MYCHEERP_PREFIX
$rmdir_cmd cheerp-compiler
$rmdir_cmd cheerp-utils
$rmdir_cmd cheerp-musl
$rmdir_cmd cheerp-libs


git clone --depth 1  https://github.com/leaningtech/cheerp-compiler
git clone --depth 1  https://github.com/leaningtech/cheerp-utils
git clone --depth 1  https://github.com/leaningtech/cheerp-musl
git clone --depth 1  https://github.com/leaningtech/cheerp-libs

cd cheerp-compiler
$cmake_cmd -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -S llvm -B build -C llvm/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS=clang -G Ninja
ninja -C build
ninja -C build install
cd ..

cd cheerp-utils
$cmake_cmd -B build -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX .
make -C build install
cd ..

cd cheerp-musl
mkdir build_genericjs
cd build_genericjs
RANLIB="$MYCHEERP_PREFIX/bin/llvm-ar s" AR="$MYCHEERP_PREFIX/bin/llvm-ar"  CC="$MYCHEERP_PREFIX/bin/clang -target cheerp" LD="$MYCHEERP_PREFIX/bin/llvm-link" CFLAGS="-Wno-int-conversion" ../configure --target=cheerp --disable-shared --prefix=$MYCHEERP_PREFIX
make clean
make -j `nproc`
make install
cd ..
mkdir build_asmjs
cd build_asmjs
RANLIB="$MYCHEERP_PREFIX/bin/llvm-ar s" AR="$MYCHEERP_PREFIX/bin/llvm-ar"  CC="$MYCHEERP_PREFIX/bin/clang -target cheerp-wasm" LD="$MYCHEERP_PREFIX/bin/llvm-link" CFLAGS="-Wno-int-conversion" ../configure --target=cheerp-wasm --disable-shared --prefix=$MYCHEERP_PREFIX
make clean
make -j `nproc`
make install
cd ../..

cd cheerp-compiler
$cmake_cmd  -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -S runtimes -B build_runtimes_genericjs -GNinja -C runtimes/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="$MYCHEERP_PREFIX/share/cmake/Modules/CheerpToolchain.cmake"
ninja -C build_runtimes_genericjs

$cmake_cmd  -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -S runtimes -B build_runtimes_wasm -GNinja -C runtimes/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="$MYCHEERP_PREFIX/share/cmake/Modules/CheerpWasmToolchain.cmake"
ninja -C build_runtimes_wasm

ninja -C build_runtimes_genericjs install
ninja -C build_runtimes_wasm install
cd ..

cd cheerp-libs
make -C webgles install INSTALL_PREFIX=$MYCHEERP_PREFIX CHEERP_PREFIX=$MYCHEERP_PREFIX
make -C wasm install INSTALL_PREFIX=$MYCHEERP_PREFIX CHEERP_PREFIX=$MYCHEERP_PREFIX
make -C stdlibs install INSTALL_PREFIX=$MYCHEERP_PREFIX CHEERP_PREFIX=$MYCHEERP_PREFIX
cd system
$cmake_cmd -B build_genericjs -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -DCMAKE_TOOLCHAIN_FILE=$MYCHEERP_PREFIX/share/cmake/Modules/CheerpToolchain.cmake -DCMAKE_BUILD_TYPE=Release .
make -C build_genericjs
make install -C build_genericjs
cd ..
cd ..

Answer 38 · 2023-06-14T11:45:03.000Z

Here is my latest build script that also include build_asmjs:

#!/bin/sh
cat $0
export PATH=$HOME/local/clang-16/bin:$PATH
export CC=clang
export CXX=clang++
cmake_cmd=mycmake
rmdir_cmd="rm -R"

MYCHEERP_PREFIX=$HOME/local/cheerp-latest
#mkdir cheerp
#cd cheerp
export CHEERP_SRC=$PWD

$rmdir_cmd $MYCHEERP_PREFIX
$rmdir_cmd cheerp-compiler
$rmdir_cmd cheerp-utils
$rmdir_cmd cheerp-musl
$rmdir_cmd cheerp-libs


git clone --depth 1  https://github.com/leaningtech/cheerp-compiler
git clone --depth 1  https://github.com/leaningtech/cheerp-utils
git clone --depth 1  https://github.com/leaningtech/cheerp-musl
git clone --depth 1  https://github.com/leaningtech/cheerp-libs

cd cheerp-compiler
$cmake_cmd -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -S llvm -B build -C llvm/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS=clang -G Ninja
ninja -C build
ninja -C build install
cd ..

cd cheerp-utils
$cmake_cmd -B build -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX .
make -C build install
cd ..

cd cheerp-musl
mkdir build_genericjs
cd build_genericjs
RANLIB="$MYCHEERP_PREFIX/bin/llvm-ar s" AR="$MYCHEERP_PREFIX/bin/llvm-ar"  CC="$MYCHEERP_PREFIX/bin/clang -target cheerp" LD="$MYCHEERP_PREFIX/bin/llvm-link" CFLAGS="-Wno-int-conversion" ../configure --target=cheerp --disable-shared --prefix=$MYCHEERP_PREFIX
make clean
make -j `nproc`
make install
cd ..
mkdir build_asmjs
cd build_asmjs
RANLIB="$MYCHEERP_PREFIX/bin/llvm-ar s" AR="$MYCHEERP_PREFIX/bin/llvm-ar"  CC="$MYCHEERP_PREFIX/bin/clang -target cheerp-wasm" LD="$MYCHEERP_PREFIX/bin/llvm-link" CFLAGS="-Wno-int-conversion" ../configure --target=cheerp-wasm --disable-shared --prefix=$MYCHEERP_PREFIX
make clean
make -j `nproc`
make install
cd ../..

cd cheerp-compiler
$cmake_cmd  -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -S runtimes -B build_runtimes_genericjs -GNinja -C runtimes/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="$MYCHEERP_PREFIX/share/cmake/Modules/CheerpToolchain.cmake"
ninja -C build_runtimes_genericjs

$cmake_cmd  -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -S runtimes -B build_runtimes_wasm -GNinja -C runtimes/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="$MYCHEERP_PREFIX/share/cmake/Modules/CheerpWasmToolchain.cmake"
ninja -C build_runtimes_wasm

ninja -C build_runtimes_genericjs install
ninja -C build_runtimes_wasm install
cd ..

cd cheerp-libs
make -C webgles install INSTALL_PREFIX=$MYCHEERP_PREFIX CHEERP_PREFIX=$MYCHEERP_PREFIX
make -C wasm install INSTALL_PREFIX=$MYCHEERP_PREFIX CHEERP_PREFIX=$MYCHEERP_PREFIX
make -C stdlibs install INSTALL_PREFIX=$MYCHEERP_PREFIX CHEERP_PREFIX=$MYCHEERP_PREFIX
cd system
$cmake_cmd -B build_genericjs -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -DCMAKE_TOOLCHAIN_FILE=$MYCHEERP_PREFIX/share/cmake/Modules/CheerpToolchain.cmake -DCMAKE_BUILD_TYPE=Release .
make -C build_genericjs
make install -C build_genericjs

$cmake_cmd -B build_asmjs -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -DCMAKE_TOOLCHAIN_FILE=$MYCHEERP_PREFIX/share/cmake/Modules/CheerpWasmToolchain.cmake -DCMAKE_BUILD_TYPE=Release .
make -C build_asmjs
make install -C build_asmjs

$cmake_cmd -B build_asmjs -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -DCMAKE_TOOLCHAIN_FILE=$MYCHEERP_PREFIX/share/cmake/Modules/CheerpWasmToolchain.cmake -DCMAKE_BUILD_TYPE=Release .
make -C build_asmjs
make install -C build_asmjs
cd ../..

Answer 39 · 2023-06-14T11:47:34.000Z

make -C stdlibs install INSTALL_PREFIX=/opt/cheerp CHEERP_PREFIX=/opt/cheerp
make -C stdlibs install_genericjs INSTALL_PREFIX=/opt/cheerp CHEERP_PREFIX=/opt/cheerp
make -C stdlibs install_asmjs INSTALL_PREFIX=/opt/cheerp CHEERP_PREFIX=/opt/cheerp

The install target will run install_genericjs and install_asmjs internally, there is no need to run them manually as well.

I have updated the READMEs and the docs website to fix the various issues

Answer 40 · 2023-06-14T11:49:54.000Z

At the end I was right that there is BAD instructions to build cheerp from sources, but you @alexp-sssup did not really checked it before instantly close this issue and we've lost a lot of time and energy to somehow have a working build script.

I hope you take the reported issues a bit more seriously, to not give a bad impression for this project !

Answer 41 · 2023-06-14T11:53:05.000Z

Looking at https://docs.leaningtech.com/cheerp/Linux-build-instructions I can see that you have updated the instructions to build latest.

But the build instructions to build stable remain broken.

Answer 42 · 2023-06-14T11:55:26.000Z

Also the install order is still not fixed:

cd cheerp-compiler
cmake -S runtimes -B build_runtimes_genericjs -GNinja -C runtimes/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="/opt/cheerp/share/cmake/Modules/CheerpToolchain.cmake"
ninja -C build_runtimes_genericjs
ninja -C build_runtimes_genericjs install

cmake -S runtimes -B build_runtimes_wasm -GNinja -C runtimes/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="/opt/cheerp/share/cmake/Modules/CheerpWasmToolchain.cmake"
ninja -C build_runtimes_wasm
ninja -C build_runtimes_wasm install
cd ..

It seems that you are not aplying attention to the whole documentation scattered in several places.

Answer 43 · 2023-06-14T11:56:49.000Z

Please refresh the page, you are most likely not seeing updated content.

You can see the source code here as confirmation: https://github.com/leaningtech/cheerp-meta/blob/master/pages/Linux-build-instructions.md

Answer 44 · 2023-06-14T11:56:59.000Z

Please publish a full working script (maybe the one that I've showed here and worked).

Answer 45 · 2023-06-14T11:59:56.000Z

Ok I can see here https://github.com/leaningtech/cheerp-meta/blob/master/pages/Linux-build-instructions.md the fixes.

Could you use a variable instead of repeating /opt/cheerp everywhere (even better a full working script like mine).

Answer 46 · 2023-06-14T12:06:26.000Z

Please refresh the docs again, a CHEERP_DEST variable is now used.

Answer 47 · 2023-06-14T12:12:41.000Z

I just refreshed and it's a lot better !

But I still do not see -DCMAKE_INSTALL_PREFIX=${CHEERP_DEST} in :

cd cheerp-compiler
cmake -S llvm -B build -C llvm/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS=clang -G Ninja
ninja -C build -j4
ninja -C build install
cd ..

Answer 48 · 2023-06-14T12:14:54.000Z

Please create a whole working script and add it to you circleci to always be sure that it works.

Answer 49 · 2023-06-14T12:16:58.000Z

Also run it yourself instead of imagining/guessing that it works !

Answer 50 · 2023-06-14T12:18:00.000Z

The instructions should now be updated to consider the prefix also for the compiler step.

We do not plan to publish an additional build script since that defeats the purpose of following the build instructions.

If a user just wants to just use Cheerp, he could use the binary packages.

If a user builds we source we assume that he wants to understand the build process. A script to run blindly would not achieve that.

Thanks for helping us making the build instruction more clear.

Answer 51 · 2023-06-14T12:30:28.000Z

Well if you add a working build script to you circleci you'll immediately know when/if it goes out of sync, otherwise you and other users will waste time again trying to figuring out what's happening.

You can reduce technical debt or allow it grow over time, make your bet !