Unable to use the library on RHEL8 - _xxsubinterpreters.

[skarndev]

Hi. I am struggling to use the library on python3.12 in a RHEL8 environment (Red Hat Linux).
dependency-injector version is 4.48.1.

The code to reproduce is:

from dependency_injector import providers, containers

The error is ModuleNotFound("_xxsubinterpreters")

During a brief investigation I found that the problem is related to Cython files containing code attempting to load an experimental module _xxsubinterpeters in Python3.12

#if CYTHON_COMPILING_IN_LIMITED_API && __PYX_LIMITED_VERSION_HEX < 0x03090000
static PY_INT64_T __Pyx_GetCurrentInterpreterId(void) {
    {
        PyObject *module = PyImport_ImportModule("_interpreters"); // 3.13+ I think
        if (!module) {
            PyErr_Clear(); // just try the 3.8-3.12 version
            module = PyImport_ImportModule("_xxsubinterpreters");
            if (!module) goto bad;
        }
        PyObject *current = PyObject_CallMethod(module, "get_current", NULL);
        Py_DECREF(module);
        if (!current) goto bad;
        if (PyTuple_Check(current)) {
            PyObject *new_current = PySequence_GetItem(current, 0);
            Py_DECREF(current);
            current = new_current;
            if (!new_current) goto bad;
        }
        long long as_c_int = PyLong_AsLongLong(current);
        Py_DECREF(current);
        return as_c_int;
    }
  bad:
    PySys_WriteStderr("__Pyx_GetCurrentInterpreterId failed. Try setting the C define CYTHON_PEP489_MULTI_PHASE_INIT=0\n");
    return -1;
}
#endif

This module is indeed available on stock Python on my local machine (MacOS, homebrew install) and is also available in python3.12 coming from Debian packages. It is not available on RHEL builds though.

I am not sure if this is an issue of Cython itself, or it is the .whl that is not compatible. If we rebuild locally, what settings should we use?

[ZipFile]

This code comes directly from Cython, so you might check with them too. It is included when Py_LIMITED_API set to 0x03080000 or lower (our case, since we build ABI3 wheels with minimal Python version being 3.8). I think we can drop support for (EOL-ed) Python 3.8 instead of trying to fix the issue.

In the mean time, try pip install dependency-injector --no-binary :all:.

[kjanko]

The same issue appears with debian buster, bullseye and bookworm. Works with the Trixie release.

[Arauf2]

Same issue on RHEL-9 - Almalinux. Using python 3.12

[ZipFile]

I've published 4.48.2 with regular wheels for 3.8 and 3.9 and ABI3 for 3.10+. This should fix the issue, since Cython code importing _xxsubinterpreters is no longer compiled-in. I've tested on redhat/ubi8:latest Docker image, not sure about different debian versions (no errors pre and post fix), though.

[slouchart]

I checked the most recent build this morning, running a docker image ubi8/python312 with the last version 4.48.2 of dependency-injector, I confirm it works, thanks.