failed to build under android (termux)
henryiii opened this issue · 19 comments
It's been about three months and I'm experiencing the same issue here. Any progress on a fix? Error log is included in the link I sent.
I have no experience with building for Android (and don't own an Android device). Happy to take help. :)
Are there any platform tags that would enable the distribution of a wheel working for termux ? Or should it be systematically build from source ?
Now since cmake is already available on the termux system 1, as suggested 2, it is probably a matter of gracefully detecting its presence and excluding the dependency.
I don't yet have a clear vision on how to scale a solution that would avoid having to implement per-package hacks ...
Footnotes
You can't distribute wheels for termux. So it would be a source build. The problem is the source build is broken on that platform.
We should provide a way to package CMake using a system CMake, but that's a specialized situation, as you can't control the CMake version. A user should be able to set a minimum version of CMake in pyproject.toml and have it respected. #227 - actually, if that was implemented and tmux was okay to distribute the python package too, that might work fairly well (though you'd still need to build without isolation).
For this specific case, the "immediate" solution would be to simply add a platform tag to avoid the cmake dependency in the user code. Up until pip 22.1 (and after the next release of pip, I believe - pypa/pip#11116), you can also turn off build isolation to control the environment. In 22.1 it will still force CMake the Python package to be present, and there's no way around it. (PS: this is the per package hacks mentioned above)
I have no experience with building for Android (and don't own an Android device). Happy to take help. :)
You could try out BlissOS or something in a VM or a spare drive/partition, but I suspect the issue would not be present on an x86 Android installation.
I expect it would be if they are QEMU emulated (don’t know if they are). But yes, likely right if they are actually a rebuild for x86.
@henryiii this appears to be a general problem on arm based machines. People are experiencing the same issue on arm64: rapidfuzz/RapidFuzz#215 and armv7: rapidfuzz/RapidFuzz#223.
I am running into similar problems with the ninja package. So for now I will use:
"cmake; platform_machine not in 'armv7l|armv6l|arm64|aarch64'",
"ninja; platform_system!='Windows' and platform_machine not in 'armv7l|armv6l|arm64|aarch64'",
and require the user to have cmake installed on the system for arm machines.
Hello experts, I am trying to install qiskit which uses cmake and ninja. I tried installing pip as well as pkg but unable get cmake and ninja installed. Any guidance much appreciated. Thanks.
Hi there experts, I was able to get ninja install work as mentioned here.
But cmake still sucks. Any suggestions would be much appreciated. Someone mentioned to pass on some flags to resolve it. Is it possible for someone to look into this.
Error line:
/CMake-src/Utilities/cmlibarchive/libarchive/archive.h:101:10: fatal error: 'android_lf.h' file not found
#include "android_lf.h" ^~~~~~~~~~~~~~
1 error generated.
Details:
[ 38%] Building C object Utilities/cmliblzma/CMakeFiles/cmliblzma.dir/liblzma/simple/simple_encoder.c.o [ 38%] Building C object Utilities/cmliblzma/CMakeFiles/cmliblzma.dir/liblzma/simple/sparc.c.o
[ 38%] Building C object Utilities/cmliblzma/CMakeFiles/cmliblzma.dir/liblzma/simple/x86.c.o
[ 38%] Linking C static library libcmliblzma.a
[ 38%] Built target cmliblzma
[ 38%] Building C object Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_acl.c.o
In file included from /data/data/com.termux/files/usr/tmp/pip-install-5jwarch6/cmake_fb7c36953597459d95ed86b96f113e3b/CMake-src/Utilities/cmlibarchive/libarchive/archive_acl.c:40:
In file included from /data/data/com.termux/files/usr/tmp/pip-install-5jwarch6/cmake_fb7c36953597459d95ed86b96f113e3b/CMake-src/Utilities/cmlibarchive/libarchive/archive_acl_private.h:35:
In file included from /data/data/com.termux/files/usr/tmp/pip-install-5jwarch6/cmake_fb7c36953597459d95ed86b96f113e3b/CMake-src/Utilities/cmlibarchive/libarchive/archive_string.h:49:
/data/data/com.termux/files/usr/tmp/pip-install-5jwarch6/cmake_fb7c36953597459d95ed86b96f113e3b/CMake-src/Utilities/cmlibarchive/libarchive/archive.h:101:10: fatal error: 'android_lf.h' file not found
#include "android_lf.h" ^~~~~~~~~~~~~~
1 error generated.
make[5]: *** [Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/build.make:76: Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_acl.c.o] Error 1
make[4]: *** [CMakeFiles/Makefile2:1990: Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/all] Error 2
make[3]: *** [Makefile:166: all] Error 2
make[2]: *** [CMakeFiles/CMakeProject-build.dir/build.make:88: CMakeProject-build-prefix/src/CMakeProject-build-stamp/CMakeProject-build-build] Error 2
make[1]: *** [CMakeFiles/Makefile2:113: CMakeFiles/CMakeProject-build.dir/all] Error 2
make: *** [Makefile:136: all] Error 2
File "/data/data/com.termux/files/usr/tmp/pip-build-env-v9s4ua_g/overlay/lib/python3.10/site-packages/skbuild/setuptools_wrap.py", line 645, in setup cmkr.make(make_args, install_target=cmake_install_target, env=env) File "/data/data/com.termux/files/usr/tmp/pip-build-env-v9s4ua_g/overlay/lib/python3.10/site-packages/skbuild/cmaker.py", line 680, in make
self.make_impl(clargs=clargs, config=config, source_dir=source_dir, install_target=install_target, env=env)
File "/data/data/com.termux/files/usr/tmp/pip-build-env-v9s4ua_g/overlay/lib/python3.10/site-packages/skbuild/cmaker.py", line 704, in make_impl
raise SKBuildError(
--------------------------------------------------------------------------------
-- Trying "Ninja" generator
--------------------------------
--------------------------- ----------------------
-----------------
------------ -------
--
--
-------
------------
-----------------
----------------------
---------------------------
--------------------------------
-- Trying "Ninja" generator - failure --------------------------------------------------------------------------------
-------------------------------------------------------------------------------- -- Trying "Unix Makefiles" generator
--------------------------------
---------------------------
----------------------
-----------------
------------
-------
--
--
-------
------------
-----------------
----------------------
---------------------------
--------------------------------
-- Trying "Unix Makefiles" generator - success
--------------------------------------------------------------------------------
Configuring Project
Working directory:
/data/data/com.termux/files/usr/tmp/pip-install-5jwarch6/cmake_fb7c36953597459d95ed86b96f113e3b/_skbuild/linux-aarch64-3.10/cmake-build
Command:
cmake /data/data/com.termux/files/usr/tmp/pip-install-5jwarch6/cmake_fb7c36953597459d95ed86b96f113e3b -G 'Unix Makefiles' -DCMAKE_INSTALL_PREFIX:PATH=/data/data/com.termux/files/usr/tmp/pip-install-5jwarch6/cmake_fb7c36953597459d95ed86b96f113e3b/_skbuild/linux-aarch64-3.10/cmake-install/src/cmake/data -DPYTHON_VERSION_STRING:STRING=3.10.6 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/data/data/com.termux/files/usr/tmp/pip-build-env-v9s4ua_g/overlay/lib/python3.10/site-packages/skbuild/resources/cmake -DPython3_EXECUTABLE:FILEPATH=/data/data/com.termux/files/usr/bin/python3 -DPython3_INCLUDE_DIR:PATH=/data/data/com.termux/files/usr/include/python3.10 -DPython3_LIBRARY:PATH=/data/data/com.termux/files/usr/lib/libpython3.10.so -DPython_EXECUTABLE:FILEPATH=/data/data/com.termux/files/usr/bin/python3 -DPython_INCLUDE_DIR:PATH=/data/data/com.termux/files/usr/include/python3.10 -DPython_LIBRARY:PATH=/data/data/com.termux/files/usr/lib/libpython3.10.so -DPYTHON_EXECUTABLE:FILEPATH=/data/data/com.termux/files/usr/bin/python3 -DPYTHON_INCLUDE_DIR:PATH=/data/data/com.termux/files/usr/include/python3.10 -DPYTHON_LIBRARY:PATH=/data/data/com.termux/files/usr/lib/libpython3.10.so -DCMAKE_BUILD_TYPE:STRING=Release
Traceback (most recent call last):
=============================DEBUG ASSISTANCE============================= If you are seeing a compilation error please try the following steps to
successfully install cmake:
1) Upgrade to the latest pip and try again. This will fix errors for most
users. See: https://pip.pypa.io/en/stable/installing/#upgrading-pip
2) If on Linux, with glibc < 2.12, you can set PIP_ONLY_BINARY=cmake in
order to retrieve the last manylinux1 compatible wheel.
3) If on Linux, with glibc < 2.12, you can cap "cmake<3.23" in your
requirements in order to retrieve the last manylinux1 compatible wheel. 4) Open an issue with the debug information that follows at
https://github.com/scikit-build/cmake-python-distributions/issues
Python: 3.10.6
platform: Linux-4.14.81-perf+-aarch64-with-libc
machine: aarch64
bits: 64
pip: n/a
setuptools: 63.4.1 scikit-build: 0.15.0
PEP517_BUILD_BACKEND=setuptools.build_meta =============================DEBUG ASSISTANCE=============================
An error occurred while building with CMake.
Command:
cmake --build . --target install --config Release --
Install target:
install
Source directory:
/data/data/com.termux/files/usr/tmp/pip-install-5jwarch6/cmake_fb7c36953597459d95ed86b96f113e3b
Working directory:
/data/data/com.termux/files/usr/tmp/pip-install-5jwarch6/cmake_fb7c36953597459d95ed86b96f113e3b/_skbuild/linux-aarch64-3.10/cmake-build
Please check the install target is valid and see CMake's output for more information.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for cmake
Failed to build cmake
ERROR: Could not build wheels for cmake, which is required to install pyproject.toml-based projects
~ $
Thanks.
Installing cmake from source requires cmake to be available. For this reason it only makes sense to install this package if a wheel is available or a newer cmake version than the one available on the system is needed. In my packages I started test for a system version of ninja/make and cmake at build time using an in-tree build backend. See here: rapidfuzz/Levenshtein@30b51bd. This way e.g. in termux users can simply install cmake from the package manager. I think it would make sense for qiskit to use a similar approach.
Thanks for suggestions @maxbachmann.
I think it would make sense for qiskit to use a similar approach.
I would link this comment on here.
At the details section of #223 (comment)
I also have that error.
I am trying to install opencv-python at termux and it seems to also install cmake for python
Any progress so far? I'm having the same error: 'android_lf.h' file not found
The path forward here is what is done in scikit-build-core, you can check for CMake and not install it if it's present via PEP 517 build hooks. In fact, you can include scikit-build-core and use the scikit_build_core.setuptools.build_api entrypoint now if you want, that injects ninja and cmake only if needed.
(It will take longer to finish scikit-build-core and move scikit-build over to it)
You will need to build both cmake and ninja from source. Once they have been built, scipy then has to be built from source (in order to use qiskit). Normal pip install scipy will build the wheel and show it is installed, but won't import. After that qiskit installs just fine. For qiskit_aer, you also need to build from source.
I have run into it yet again AD 2024 - a kind of deja vu.
Ugly solution:
git clone https://github.com/libarchive/libarchive
Locate where the missing file is:
~/downloads/cmake-python-distributions $ find . -name android_lf.h
./libarchive/contrib/android/include/android_lf.h
~/downloads/cmake-python-distributions $
2a. Change it in the .h files:
/data/data/com.termux/files/home/downloads/cmake-python-distributions/CMake-src/Utilities/cmlibarchive/libarchive/archive.h
and
/data/data/com.termux/files/home/downloads/cmake-python-distributions/CMake-src/Utilities/cmlibarchive/libarchive/archive_entry.h
- e.g.
99 /* Large file support for Android */
100 #if defined(__LIBARCHIVE_BUILD) && defined(__ANDROID__)
101 #include "/data/data/com.termux/files/home/downloads/cmake-python-distributions/libarchive/contrib/android/include/android_lf.h"
102 #endif
or
2b.
Add into CMakeLists.txt in the same directory as the libarchive source:
include_directories(${CMAKE_CURRENT_SOURCE_DIR}/contrib/android/include)
or
2c.
find . -type f -print0 | xargs -0 -r sed -i 's/android_lf\.h/\/data\/data\/com\.termux\/files\/home\/downloads\/cmake-python-distributions\/libarchive\/contrib\/android\/include\/android_lf\.h/g'
if you know what it does.
[20/656] Building C object Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_read_disk_posix.c.o
FAILED: Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_read_disk_posix.c.o
/data/data/com.termux/files/usr/bin/clang -DHAVE_CONFIG_H -DLZMA_API_STATIC -D_XOPEN_SOURCE=600 -I/data/data/com.termux/files/home/downloads/cmake-python-distributions/CMake-src/Utilities/cmlibarchive/libarchive -I/data/data/com.termux/files/home/downloads/cmake-python-distributions/build/py3-none-linux_aarch64/CMakeProject-build/Utilities/cmlibarchive -I/data/data/com.termux/files/home/downloads/cmake-python-distributions/build/py3-none-linux_aarch64/CMakeProject-build/Utilities -I/data/data/com.termux/files/home/downloads/cmake-python-distributions/CMake-src/Utilities/cmbzip2 -I/data/data/com.termux/files/home/downloads/cmake-python-distributions/CMake-src/Utilities/cmliblzma/liblzma/api -I/data/data/com.termux/files/home/downloads/cmake-python-distributions/CMake-src/Utilities/cmzstd -I/data/data/com.termux/files/home/downloads/cmake-python-distributions/CMake-src/Utilities/cmlibarchive/contrib/android/include -isystem /data/data/com.termux/files/home/downloads/cmake-python-distributions/CMake-src/Utilities/std -isystem /data/data/com.termux/files/home/downloads/cmake-python-distributions/CMake-src/Utilities -D_POSIX_C_SOURCE=199506L -D_POSIX_SOURCE=1 -D_SVID_SOURCE=1 -D_BSD_SOURCE=1 -w -O3 -DNDEBUG -std=gnu11 -MD -MT Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_read_disk_posix.c.o -MF Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_read_disk_posix.c.o.d -o Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_read_disk_posix.c.o -c /data/data/com.termux/files/home/downloads/cmake-python-distributions/CMake-src/Utilities/cmlibarchive/libarchive/archive_read_disk_posix.c
/data/data/com.termux/files/home/downloads/cmake-python-distributions/CMake-src/Utilities/cmlibarchive/libarchive/archive_read_disk_posix.c:2080:6: error: call to undeclared function 'futimesat'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
2080 | if (futimesat(tree_current_dir_fd(t), rt->name, times) == 0)
| ^
/data/data/com.termux/files/home/downloads/cmake-python-distributions/CMake-src/Utilities/cmlibarchive/libarchive/archive_read_disk_posix.c:2080:6: note: did you mean 'utimensat'?
/data/data/com.termux/files/usr/include/sys/stat.h:204:5: note: 'utimensat' declared here
204 | int utimensat(int __dir_fd, const char* __BIONIC_COMPLICATED_NULLNESS __path, const struct timespec __times[_Nullable 2], int __flags);
| ^
/data/data/com.termux/files/home/downloads/cmake-python-distributions/CMake-src/Utilities/cmlibarchive/libarchive/archive_read_disk_posix.c:2084:6: error: call to undeclared function 'lutimes'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
2084 | if (lutimes(rt->name, times) != 0)
| ^
/data/data/com.termux/files/home/downloads/cmake-python-distributions/CMake-src/Utilities/cmlibarchive/libarchive/archive_read_disk_posix.c:2084:6: note: did you mean 'utimes'?
/data/data/com.termux/files/usr/include/sys/time.h:47:5: note: 'utimes' declared here
47 | int utimes(const char* _Nonnull __path, const struct timeval __times[_Nullable 2]);
| ^
2 errors generated.
for which see my remark here:
libarchive/libarchive#2214
PS. See also: https://gitlab.kitware.com/cmake/cmake/-/issues/21399 - supposedly a perennial problem.
A solution: termux/termux-packages#10065 (comment) - pro memoriam for myself, as I keep finding that bug once again.