fsspec/s3fs

s3fs package cannot be installed in a python 3.12 env

xtianpoli opened this issue · 4 comments

The problem is related to the aiohttp dependency. See below a fragment of the trace:

× Building wheel for aiohttp (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [173 lines of output]
*********************
* Accelerated build *
*********************
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-312
creating build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/http.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/client_exceptions.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/web.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/log.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/payload_streamer.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/abc.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/base_protocol.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/multipart.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/web_routedef.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/web_app.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/http_writer.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/cookiejar.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/tcp_helpers.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/test_utils.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/web_runner.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/web_request.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/web_middlewares.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/web_response.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/web_log.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/client.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/client_proto.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/pytest_plugin.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/client_ws.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/resolver.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/http_websocket.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/web_ws.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/hdrs.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/payload.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/http_exceptions.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/locks.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/tracing.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/web_server.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/web_fileresponse.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/client_reqrep.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/helpers.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/formdata.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/streams.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/init.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/web_urldispatcher.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/connector.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/http_parser.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/typedefs.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/web_exceptions.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/web_protocol.py -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/worker.py -> build/lib.linux-x86_64-cpython-312/aiohttp
running egg_info
writing aiohttp.egg-info/PKG-INFO
writing dependency_links to aiohttp.egg-info/dependency_links.txt
writing requirements to aiohttp.egg-info/requires.txt
writing top-level names to aiohttp.egg-info/top_level.txt
reading manifest file 'aiohttp.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'aiohttp' anywhere in distribution
warning: no previously-included files matching '.pyc' found anywhere in distribution
warning: no previously-included files matching '
.pyd' found anywhere in distribution
warning: no previously-included files matching '.so' found anywhere in distribution
warning: no previously-included files matching '
.lib' found anywhere in distribution
warning: no previously-included files matching '.dll' found anywhere in distribution
warning: no previously-included files matching '
.a' found anywhere in distribution
warning: no previously-included files matching '.obj' found anywhere in distribution
warning: no previously-included files found matching 'aiohttp/
.html'
no previously-included directories found matching 'docs/_build'
adding license file 'LICENSE.txt'
writing manifest file 'aiohttp.egg-info/SOURCES.txt'
copying aiohttp/_cparser.pxd -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/_find_header.pxd -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/_headers.pxi -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/_helpers.pyi -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/_helpers.pyx -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/_http_parser.pyx -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/_http_writer.pyx -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/_websocket.pyx -> build/lib.linux-x86_64-cpython-312/aiohttp
copying aiohttp/py.typed -> build/lib.linux-x86_64-cpython-312/aiohttp
creating build/lib.linux-x86_64-cpython-312/aiohttp/.hash
copying aiohttp/.hash/_cparser.pxd.hash -> build/lib.linux-x86_64-cpython-312/aiohttp/.hash
copying aiohttp/.hash/_find_header.pxd.hash -> build/lib.linux-x86_64-cpython-312/aiohttp/.hash
copying aiohttp/.hash/_helpers.pyi.hash -> build/lib.linux-x86_64-cpython-312/aiohttp/.hash
copying aiohttp/.hash/_helpers.pyx.hash -> build/lib.linux-x86_64-cpython-312/aiohttp/.hash
copying aiohttp/.hash/_http_parser.pyx.hash -> build/lib.linux-x86_64-cpython-312/aiohttp/.hash
copying aiohttp/.hash/_http_writer.pyx.hash -> build/lib.linux-x86_64-cpython-312/aiohttp/.hash
copying aiohttp/.hash/_websocket.pyx.hash -> build/lib.linux-x86_64-cpython-312/aiohttp/.hash
copying aiohttp/.hash/hdrs.py.hash -> build/lib.linux-x86_64-cpython-312/aiohttp/.hash
running build_ext
building 'aiohttp._websocket' extension
creating build/temp.linux-x86_64-cpython-312
creating build/temp.linux-x86_64-cpython-312/aiohttp
gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12 -c aiohttp/_websocket.c -o build/temp.linux-x86_64-cpython-312/aiohttp/_websocket.o
aiohttp/_websocket.c: In function ‘__pyx_pf_7aiohttp_10_websocket__websocket_mask_cython’:
aiohttp/_websocket.c:1475:3: warning: ‘Py_OptimizeFlag’ is deprecated [-Wdeprecated-declarations]
1475 | if (unlikely(!Py_OptimizeFlag)) {
| ^~
In file included from /opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12/Python.h:48,
from aiohttp/_websocket.c:6:
/opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12/cpython/pydebug.h:13:37: note: declared here
13 | Py_DEPRECATED(3.12) PyAPI_DATA(int) Py_OptimizeFlag;
| ^~~~~~~~~~~~~~~
aiohttp/_websocket.c: In function ‘__Pyx_get_tp_dict_version’:
aiohttp/_websocket.c:2680:5: warning: ‘ma_version_tag’ is deprecated [-Wdeprecated-declarations]
2680 | return likely(dict) ? __PYX_GET_DICT_VERSION(dict) : 0;
| ^~~~~~
In file included from /opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12/dictobject.h:90,
from /opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12/Python.h:61,
from aiohttp/_websocket.c:6:
/opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12/cpython/dictobject.h:22:34: note: declared here
22 | Py_DEPRECATED(3.12) uint64_t ma_version_tag;
| ^~~~~~~~~~~~~~
aiohttp/_websocket.c: In function ‘__Pyx_get_object_dict_version’:
aiohttp/_websocket.c:2692:5: warning: ‘ma_version_tag’ is deprecated [-Wdeprecated-declarations]
2692 | return (dictptr && dictptr) ? __PYX_GET_DICT_VERSION(dictptr) : 0;
| ^~~~~~
In file included from /opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12/dictobject.h:90,
from /opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12/Python.h:61,
from aiohttp/_websocket.c:6:
/opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12/cpython/dictobject.h:22:34: note: declared here
22 | Py_DEPRECATED(3.12) uint64_t ma_version_tag;
| ^~~~~~~~~~~~~~
aiohttp/_websocket.c: In function ‘__Pyx_object_dict_version_matches’:
aiohttp/_websocket.c:2696:5: warning: ‘ma_version_tag’ is deprecated [-Wdeprecated-declarations]
2696 | if (unlikely(!dict) || unlikely(tp_dict_version != __PYX_GET_DICT_VERSION(dict)))
| ^~
In file included from /opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12/dictobject.h:90,
from /opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12/Python.h:61,
from aiohttp/_websocket.c:6:
/opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12/cpython/dictobject.h:22:34: note: declared here
22 | Py_DEPRECATED(3.12) uint64_t ma_version_tag;
| ^~~~~~~~~~~~~~
aiohttp/_websocket.c: In function ‘__Pyx_CLineForTraceback’:
aiohttp/_websocket.c:2741:9: warning: ‘ma_version_tag’ is deprecated [-Wdeprecated-declarations]
2741 | __PYX_PY_DICT_LOOKUP_IF_MODIFIED(
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12/dictobject.h:90,
from /opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12/Python.h:61,
from aiohttp/_websocket.c:6:
/opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12/cpython/dictobject.h:22:34: note: declared here
22 | Py_DEPRECATED(3.12) uint64_t ma_version_tag;
| ^~~~~~~~~~~~~~
aiohttp/_websocket.c:2741:9: warning: ‘ma_version_tag’ is deprecated [-Wdeprecated-declarations]
2741 | __PYX_PY_DICT_LOOKUP_IF_MODIFIED(
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12/dictobject.h:90,
from /opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12/Python.h:61,
from aiohttp/_websocket.c:6:
/opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12/cpython/dictobject.h:22:34: note: declared here
22 | Py_DEPRECATED(3.12) uint64_t ma_version_tag;
| ^~~~~~~~~~~~~~
aiohttp/_websocket.c: In function ‘__Pyx_PyInt_As_long’:
aiohttp/_websocket.c:3042:53: error: ‘PyLongObject’ {aka ‘struct _longobject’} has no member named ‘ob_digit’
3042 | const digit
digits = ((PyLongObject
)x)->ob_digit;
| ^~
aiohttp/_websocket.c:3097:53: error: ‘PyLongObject’ {aka ‘struct _longobject’} has no member named ‘ob_digit’
3097 | const digit* digits = ((PyLongObject*)x)->ob_digit;
| ^~
aiohttp/_websocket.c: In function ‘__Pyx_PyInt_As_int’:
aiohttp/_websocket.c:3238:53: error: ‘PyLongObject’ {aka ‘struct _longobject’} has no member named ‘ob_digit’
3238 | const digit* digits = ((PyLongObject*)x)->ob_digit;
| ^~
aiohttp/_websocket.c:3293:53: error: ‘PyLongObject’ {aka ‘struct _longobject’} has no member named ‘ob_digit’
3293 | const digit* digits = ((PyLongObject*)x)->ob_digit;
| ^~
aiohttp/_websocket.c: In function ‘__Pyx_PyIndex_AsSsize_t’:
aiohttp/_websocket.c:3744:45: error: ‘PyLongObject’ {aka ‘struct _longobject’} has no member named ‘ob_digit’
3744 | const digit* digits = ((PyLongObject*)b)->ob_digit;
| ^~
error: command '/usr/bin/gcc' failed with exit code 1

NB: a related issue was already opened on aiohttp aio-libs/aiohttp#7765

It seems you will need to wait for py3.12 wheels to appear at https://pypi.org/project/aiohttp/#files

conda-forge seems to have them:
https://anaconda.org/conda-forge/aiohttp/files

Jamim commented

Hi everyone,

For the sake of transparency, I'd like to mention that there is a corresponding issue: aio-libs/aiohttp#7739

@xtianpoli part of the log you posted does not include it but the problem is that you're trying to compile an aiohttp version that is not compatible with Python 3.12. The recent one ships pre-compiled wheels already so you can close this now.