msgpack/msgpack-python

Python 3.13: compile error on Ubuntu 20.04

tristanlatr opened this issue · 5 comments

Impossible to build msgpack on python 3.13 because of what looks like an deprecation warning

  × Building wheel for msgpack (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [45 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-313
      creating build/lib.linux-x86_64-cpython-313/msgpack
      copying msgpack/fallback.py -> build/lib.linux-x86_64-cpython-313/msgpack
      copying msgpack/__init__.py -> build/lib.linux-x86_64-cpython-313/msgpack
      copying msgpack/exceptions.py -> build/lib.linux-x86_64-cpython-313/msgpack
      copying msgpack/ext.py -> build/lib.linux-x86_64-cpython-313/msgpack
      running build_ext
      building 'msgpack._cmsgpack' extension
      creating build/temp.linux-x86_64-cpython-313
      creating build/temp.linux-x86_64-cpython-313/msgpack
      gcc -pthread -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I. -I/home/runner/work/pydoctor/pydoctor/.tox/test/include -I/opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13 -c msgpack/_cmsgpack.cpp -o build/temp.linux-x86_64-cpython-313/msgpack/_cmsgpack.o
      msgpack/_cmsgpack.cpp:1366:72: warning: ‘Py_UNICODE’ is deprecated [-Wdeprecated-declarations]
       1366 | static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u)
            |                                                                        ^
      In file included from /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/unicodeobject.h:1014,
                       from /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/Python.h:65,
                       from msgpack/_cmsgpack.cpp:16:
      /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/cpython/unicodeobject.h:10:37: note: declared here
         10 | Py_DEPRECATED(3.13) typedef wchar_t Py_UNICODE;
            |                                     ^~~~~~~~~~
      msgpack/_cmsgpack.cpp: In function ‘size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE*)’:
      msgpack/_cmsgpack.cpp:1368:23: warning: ‘Py_UNICODE’ is deprecated [-Wdeprecated-declarations]
       1368 |     const Py_UNICODE *u_end = u;
            |                       ^~~~~
      In file included from /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/unicodeobject.h:1014,
                       from /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/Python.h:65,
                       from msgpack/_cmsgpack.cpp:16:
      /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/cpython/unicodeobject.h:10:37: note: declared here
         10 | Py_DEPRECATED(3.13) typedef wchar_t Py_UNICODE;
            |                                     ^~~~~~~~~~
      In file included from /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/internal/pycore_code.h:461,
                       from /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/internal/pycore_frame.h:13,
                       from msgpack/_cmsgpack.cpp:22559:
      /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/internal/pycore_backoff.h: In function ‘_Py_BackoffCounter make_backoff_counter(uint16_t, uint16_t)’:
      /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/internal/pycore_backoff.h:47:44: warning: narrowing conversion of ‘backoff’ from ‘uint16_t’ {aka ‘short unsigned int’} to ‘unsigned char:4’ [-Wnarrowing]
         47 |     return (_Py_BackoffCounter){.backoff = backoff, .value = value};
            |                                            ^~~~~~~
      /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/internal/pycore_backoff.h:47:67: error: too many initializers for ‘_Py_BackoffCounter’
         47 |     return (_Py_BackoffCounter){.backoff = backoff, .value = value};
            |                                                                   ^
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

It started failing at 3.13.0-alpha.6, everything works on 3.13.0-alpha.5.

I suspect that the unicode deprecation warning does not fail the build, it is just a warning.

But that is an error:

      In file included from /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/internal/pycore_code.h:461,
                       from /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/internal/pycore_frame.h:13,
                       from msgpack/_cmsgpack.cpp:22559:
      /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/internal/pycore_backoff.h: In function ‘_Py_BackoffCounter make_backoff_counter(uint16_t, uint16_t)’:
      /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/internal/pycore_backoff.h:47:44: warning: narrowing conversion of ‘backoff’ from ‘uint16_t’ {aka ‘short unsigned int’} to ‘unsigned char:4’ [-Wnarrowing]
         47 |     return (_Py_BackoffCounter){.backoff = backoff, .value = value};
            |                                            ^~~~~~~
      /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/internal/pycore_backoff.h:47:67: error: too many initializers for ‘_Py_BackoffCounter’
         47 |     return (_Py_BackoffCounter){.backoff = backoff, .value = value};
            |                                                                   ^
      error: command '/usr/bin/gcc' failed with exit code 1

Considering this is built from Cython source (.pyx), maybe the Cython version used to cythonize that does not yet support that Python version.

This is not relating to Cython. Maybe, default g++ in ubuntu-20.04 Action Runner doesn't support compiling this header file.

Uh, this is relating to Cython, because Cython depending on Python internal too much.
Cython imports pycore_frame for traceback support.

/* AddTraceback */
#include "compile.h"
#include "frameobject.h"
#include "traceback.h"
#if PY_VERSION_HEX >= 0x030b00a6 && !CYTHON_COMPILING_IN_LIMITED_API
  #ifndef Py_BUILD_CORE
    #define Py_BUILD_CORE 1
  #endif
  #include "internal/pycore_frame.h"
#endif
#if CYTHON_COMPILING_IN_LIMITED_API
static PyObject *__Pyx_PyCode_Replace_For_AddTraceback(PyObject *code, PyObject *scratch_dict,
                                                       PyObject *firstlineno, PyObject *name) {
    PyObject *replace = NULL;
...

It cause compile error on old g++.

inada-n@ubuntu20:~$ cat foo.cpp
#define Py_BUILD_CORE 1
#include <Python.h>

#include "frameobject.h"
#include "internal/pycore_frame.h"

int main() {}

inada-n@ubuntu20:~$ g++-10 -std=c++17 -I include/python3.13 foo.cpp
In file included from include/python3.13/internal/pycore_code.h:461,
                 from include/python3.13/internal/pycore_frame.h:13,
                 from foo.cpp:5:
include/python3.13/internal/pycore_backoff.h: In function ‘_Py_BackoffCounter make_backoff_counter(uint16_t, uint16_t)’:
include/python3.13/internal/pycore_backoff.h:47:44: warning: narrowing conversion of ‘backoff’ from ‘uint16_t’ {aka ‘short unsigned int’} to ‘unsigned char:4’ [-Wnarrowing]
   47 |     return (_Py_BackoffCounter){.backoff = backoff, .value = value};
      |                                            ^~~~~~~
include/python3.13/internal/pycore_backoff.h:47:67: error: too many initializers for ‘_Py_BackoffCounter’
   47 |     return (_Py_BackoffCounter){.backoff = backoff, .value = value};
      |                                                                   ^

On Ubuntu 22.04, this error doesn't happen. Ubuntu 22.04 uses gcc 13.

There is nothing we can do here. You can:

  • Install gcc version used by Ubuntu 22.04, or
  • Update your CI to Ubuntu 22.04