msys2-contrib/cpython-mingw

CI: cross build some modules requiring external libraries

lazka opened this issue · 2 comments

lazka commented

The following modules don't build in the cross CI jobs currently because external dependencies are missing:

  • Modules/pyexpat.c:5:10: fatal error: 'expat.h' file not found
  • Modules/_elementtree.c:3033:10: fatal error: 'expat.h' file not found
  • Modules/_decimal/_decimal.c:35:10: fatal error: 'mpdecimal.h' file not found
  • Modules/_ctypes/_ctypes.c:118:10: fatal error: 'ffi.h' file not found
  • _sqlite3 is disabled somehow

We could build the external dependencies, then build Python with them, dist them, and test that the modules can be imported in the "cross-gcc-x86_64-test" job.

This is motivated by #145 (comment)

jeremyd2019 commented

When I was testing cross-compilation within msys2 using msys2/MINGW-packages#8762 I was able to build all the modules that were included in the MINGW-packages mingw-w64-python package. But that was with 3.10, I have not tried that with 3.11 yet.

jeremyd2019 commented

I just tried with 3.11 (before #145 was merged) and successfully got this summary from setup.py:

The necessary bits to build these optional modules were not found:
_crypt                _dbm                  _gdbm
_posixshmem           _posixsubprocess      _uuid
fcntl                 grp                   nis
ossaudiodev           readline              resource
spwd                  syslog                termios
To find the necessary bits, look in setup.py in detect_modules() for the module's name.

Which looks correct. In case it helps, here's the script I used (needed minor tweaks for python 3.11 and the PKG_CONFIG_LIBDIR variable I just learned about)

Python cross-compile script for MSYS2 
#!/bin/bash -ex

TARGET=${TARGET:-aarch64-w64-mingw32}
PREFIX=${PREFIX:-/clangarm64}

export PKG_CONFIG_LIBDIR=${PREFIX}/lib/pkgconfig:${PREFIX}/share/pkgconfig
export CC=${TARGET}-clang
export CXX=${CC}++
pacman -S --asdeps --needed --noconfirm --noprogressbar autoconf autoconf-archive automake-wrapper

git clone --depth 1 --single-branch https://github.com/msys2-contrib/cpython-mingw.git
cd cpython-mingw
autoreconf -vfi

mkdir _build && cd _build
../configure \
        --prefix=${PREFIX} \
        --build=${MINGW_CHOST} \
        --host=${TARGET} \
        --with-build-python=python \
        --enable-shared \
        --with-system-expat \
        --with-system-ffi \
        --with-system-libmpdec \
        --without-ensurepip \
        --enable-loadable-sqlite-extensions \
        --with-tzpath=${PREFIX}/share/zoneinfo

make -j8
make -j1 install DESTDIR=python_pkgdir