Support for the termux android app
commiyou opened this issue · 9 comments
Termux does not work well with binary wheels. I havn't find a way to make it work on termux. However, it works on ubuntu inside termux. First, you need to install the AnLinux app, then install a chrooted ubuntu, everything should work out of the box.
I would like to install revChatGPT on Termux, which depends on curl_cffi. I manually installed curl_cffi with the following modifications:
- Installed
rustandbinutils. - Modified
preprocess/download_so.py, settingLIBDIR = '/data/data/com.termux/files/usr'. - Commented out
delocate-wheel dist/*.whlin the makefile. - Finally, generated
dist/curl_cffi-0.5.7-cp37-abi3-linux_aarch64.whland installed it using pip.
I hope this information is helpful.
I would like to install revChatGPT on Termux, which depends on curl_cffi. I manually installed curl_cffi with the following modifications:
- Installed
rustandbinutils.- Modified
preprocess/download_so.py, settingLIBDIR = '/data/data/com.termux/files/usr'.- Commented out
delocate-wheel dist/*.whlin the makefile.- Finally, generated
dist/curl_cffi-0.5.7-cp37-abi3-linux_aarch64.whland installed it using pip.I hope this information is helpful.
@commiyou can you share the curl-cffi whl file? It doesn't work for me.
@Almaz-Kabirov you also need to compile the corresponding libcurl-impersonate-chrome.so.4.8.0 shared library via curl-impersonate and place it in Termux's /data/data/com.termux/file/usr/lib folder, then only edit the makefile for curl_cffi and can compile the curl_cffi.whl file.
Check my discussion in install gpt4free in Termux to get started. I can send the build script later when I have time.
@George-Seven >I can send the build script later when I have time.
Faced with the same issues. Kindly remind for building script
According to this thread, it seems that the default manylinux wheel if not suitable for termux environment. I will build special wheels and put them in GitHub releases in the future.
@Ivan-Firefly this build script is originally meant for creating a single deb file that installs gpt4free for Termux.
It builds all the dependencies like libcurl-impersonate-chrome.so.4.8.0(curl-impersonate-chrome) and curl_cffi.whl, and then bundles them together into a single deb file which can be installed on any same architecture devices using -
pkg install ./path/to/python-g4f.deb
You can install the output deb file which will setup all dependencies itself, or you can modify the script to build as per needs.
You can find all the built artifacts inside the curl-impersonate/output folder, including the built curl_cffi.whl.
To build it nano build_g4f_deb.sh and copy/paste the script and CTRL + X + Y + ENTER to save it and then bash build_g4f_deb.sh
#!/data/data/com.termux/files/usr/bin/env bash
set -e
out_dir="output"
out_dir="$(basename "${out_dir}")"
manifest_json='
{
"control": {
"Package": "PLACEHOLDER",
"Version": "PLACEHOLDER",
"Architecture": "PLACEHOLDER",
"Maintainer": "PLACEHOLDER",
"Pre-Depends": [
"PLACEHOLDER"
],
"Homepage": "PLACEHOLDER",
"Description": "PLACEHOLDER"
},
"data_files": {
"PLACEHOLDER": {
"source": "PLACEHOLDER"
}
}
}
'
termux-wake-lock &>/dev/null || true
yes | pkg upgrade -y
pkg update -y
pkg reinstall -y openssl openssl-static
pkg install -y build-essential pkg-config cmake ninja curl autoconf automake libtool golang jq binutils rust git termux-elf-cleaner python-pip python-cryptography libgsasl libpsl termux-create-package
rm -rf curl-impersonate
git clone --depth=1 https://github.com/lwthiker/curl-impersonate
cd curl-impersonate
mkdir "${out_dir}"
manifest_json="$(jq '.data_files = {"share/python-g4f/bin/curl-impersonate-chrome": { "source": "curl-impersonate-chrome" }}' <<< "$manifest_json")"
for i in $(find chrome -type f -name "curl_*"); do
cp -rf "${i}" "${out_dir}"
i="$(basename "${i}")"
manifest_json="$(jq ".data_files += {\"share/python-g4f/bin/${i}\": { \"source\": \"${i}\" }}" <<< "$manifest_json")"
done
mkdir build
cd build
clear || true
../configure --prefix="$PREFIX"
curl_version="$(sed -En 's/^CURL_VERSION := (.+)/\1/p' Makefile)"
sed -i '/\.\/configure \$\$config_flags LIBS="$$add_libs"/q' Makefile
sed -i '/\.\/configure \$\$config_flags LIBS="\$\$add_libs"/i \\tfind . -maxdepth 1 -type f -exec sed -i "s#/etc/ssl/cert.pem#$(PREFIX)\/etc\/tls\/cert.pem#g" {} +' Makefile
sed -i '/\.\/configure \$\$config_flags LIBS="\$\$add_libs"/i \\tfind . -maxdepth 1 -type f -exec sed -i "s#/etc/ssl/certs#$(PREFIX)/etc/tls#g" {} +' Makefile
sed -i '/\.\/configure \$\$config_flags LIBS="\$\$add_libs"/i \\tfind . -maxdepth 1 -type f -exec sed -i "s/ca-certificates\.crt/cert\.pem/g" {} +' Makefile
sed -i '/\.\/configure \$\$config_flags LIBS="\$\$add_libs"/a \\trm -r "$(PREFIX)\/lib\/"libssl.* "$(PREFIX)\/lib\/"libcrypto.* "$(PREFIX)\/include\/openssl"' Makefile
sed -i '/cd boringssl\//a \\tsed -i '\''s\/-Werror\/\/g'\'' CMakeLists.txt' Makefile
sed -i '/cd boringssl\//a \\tsed -i '\''s\/\&\& \!defined(OPENSSL_ANDROID)\/\/g'\'' ssl\/test\/handshake_util.h' Makefile
set +e
make chrome-build
pkg reinstall -y openssl openssl-static
set -e
cd .."/${out_dir}"
mv -f .."/build/${curl_version}/src/.libs/curl-impersonate-chrome" .
mv -f .."/build/${curl_version}/lib/.libs/libcurl-impersonate-chrome.a" .
mv -f .."/build/${curl_version}/lib/.libs/libcurl-impersonate-chrome.so.4.8.0" .
strip -s * &>/dev/null || true
termux-elf-cleaner * &>/dev/null || true
ln -s libcurl-impersonate-chrome.so.4.8.0 libcurl-impersonate-chrome.so
ln -s libcurl-impersonate-chrome.so.4.8.0 libcurl-impersonate-chrome.so.4
cp -pf curl-impersonate-chrome "$PREFIX/bin"
cp -pf libcurl-impersonate-chrome.so.4.8.0 "$PREFIX/lib"
$(cd "$PREFIX/lib"; ln -sf libcurl-impersonate-chrome.so.4.8.0 libcurl-impersonate-chrome.so; ln -sf libcurl-impersonate-chrome.so.4.8.0 libcurl-impersonate-chrome.so.4)
cd ../build
git clone --depth=1 https://github.com/yifeikong/curl_cffi
cd curl_cffi
# Needs fix, but skipping for now
sed -i -E 's/(.*MAX_RECV_SPEED_LARGE.*)/# \1/' curl_cffi/requests/session.py
# Fix Makefile
sed -i '/delocate-wheel/d' Makefile
sed -i 's/delocate//g' Makefile
sed -i '/urlretrieve(/d' scripts/build.py
set +e
termux-wake-lock &>/dev/null
CARGO_BUILD_TARGET="$(rustc -Vv | grep "host" | awk '{print $2}')" pip install -U maturin || true
retry=1
while ! pip show maturin &>/dev/null && ((retry < 10)); do
termux-wake-lock &>/dev/null
sleep 5
CARGO_BUILD_TARGET="$(rustc -Vv | grep "host" | awk '{print $2}')" pip install -U maturin || true
((retry++))
done
set -e
rm -rf "$PREFIX/include/curl"
mv -f .."/${curl_version}/include/curl" "$PREFIX/include"
set +e
make build
rm -rf "$PREFIX/include/curl"
pkg reinstall -y libcurl
set -e
mv -f "dist/"curl_cffi-*.whl ../.."/${out_dir}"
mkdir ../python-g4f
cd ../python-g4f
postinst='#!/data/data/com.termux/files/usr/bin/env bash
set -e
mv -f "$PREFIX/share/python-g4f/bin/"* "$PREFIX/bin"
mv -f "$PREFIX/share/python-g4f/lib/"* "$PREFIX/lib"
ln -sf "$PREFIX/lib/libcurl-impersonate-chrome.so.4.8.0" "$PREFIX/lib/libcurl-impersonate-chrome.so"
ln -sf "$PREFIX/lib/libcurl-impersonate-chrome.so.4.8.0" "$PREFIX/lib/libcurl-impersonate-chrome.so.4"
pip install -U --force-reinstall "$PREFIX/share/python-g4f/"curl_cffi-*.whl || true
retry=1
while ! pip show curl_cffi &>/dev/null && ((retry < 10)); do
sleep 5
pip install -U --force-reinstall "$PREFIX/share/python-g4f/"curl_cffi-*.whl || true
((retry++))
done
rm -rf "$PREFIX/share/python-g4f" &>/dev/null
pip install -U cython flask
CFLAGS="-Wno-error=incompatible-function-pointer-types" pip install -U lxml
set +e
pip install -U g4f || true
retry=1
while ! pip show g4f &>/dev/null && ((retry < 10)); do
sleep 5
pip install -U g4f || true
((retry++))
done
if ! pip show g4f &>/dev/null; then
exit 1
fi
set -e
echo -e "\n Try testing it -"
echo -e "\n g4f gui -port 4447"
echo -e "\n And open the url http://127.0.0.1:4447/chat in the browser"
echo -e ""
'
echo "$postinst" > postinst
curl_cffi_whl="$(basename "$(ls ../.."/${out_dir}/"curl_cffi-*.whl | head -n 1)")"
manifest_json="$(jq '.control.Package = "python-g4f"' <<< "$manifest_json")"
manifest_json="$(jq '.control.Maintainer = "@George-Seven"' <<< "$manifest_json")"
manifest_json="$(jq '.control.["Pre-Depends"] = ["python-pip", "python-numpy", "python-pillow", " binutils", "libxml2", "libxslt", "libgsasl", "libpsl", "rust"]' <<< "$manifest_json")"
manifest_json="$(jq '.control.Homepage = "https://github.com/xtekky/gpt4free"' <<< "$manifest_json")"
manifest_json="$(jq '.control.Description = "Pre-compiled dependencies for installing the g4f (gpt4free) Python PyPi package, which cannot be compiled out-of-the-box in Termux. The g4f PyPi package allows access to various collection of powerful language models, like GPT-4, for free in Python"' <<< "$manifest_json")"
manifest_json="$(jq '.control_files_dir = "."' <<< "$manifest_json")"
manifest_json="$(jq ".deb_dir = \"../../${out_dir}\"" <<< "$manifest_json")"
manifest_json="$(jq ".files_dir = \"../../${out_dir}\"" <<< "$manifest_json")"
manifest_json="$(jq '.data_files += {"share/python-g4f/lib/libcurl-impersonate-chrome.a": { "source": "libcurl-impersonate-chrome.a" }}' <<< "$manifest_json")"
manifest_json="$(jq '.data_files += {"share/python-g4f/lib/libcurl-impersonate-chrome.so.4.8.0": { "source": "libcurl-impersonate-chrome.so.4.8.0" }}' <<< "$manifest_json")"
manifest_json="$(jq ".data_files += {\"share/python-g4f/${curl_cffi_whl}\": { \"source\": \"${curl_cffi_whl}\" }}" <<< "$manifest_json")"
echo "$manifest_json" | jq . > manifest.json
termux-create-package --pkg-arch "$(dpkg --print-architecture)" --pkg-version "$(curl -sL https://pypi.org/pypi/g4f/json | jq -r .info.version)" manifest.json
set +e
shopt -s extglob
$(cd ../..; rm -rf !("${out_dir}") .* &>/dev/null)
echo -e "\nTermux debian package created, you can install it with -\n\n pkg install -y \"$(cd ../.."/${out_dir}"; echo "$(pwd)/$(basename $(ls python-g4f*.deb | head -n 1))")\"\n\n"
Did anyone encountered libpthread not found error?