"AttributeError: cython_sources" with Cython 3.0.0a10
The-Compiler opened this issue Β· 104 comments
When attempting to install PyYAML from sources with Cython 3.0.0a10, e.g. like so:
pip install --pre -v git+https://github.com/yaml/pyyaml.git
the install fails with:
Using pip 21.3.1 from /home/florian/tmp/toxtest/.venv/lib/python3.10/site-packages/pip (python 3.10)
Collecting git+https://github.com/yaml/pyyaml.git
Cloning https://github.com/yaml/pyyaml.git to /tmp/pip-req-build-rdv2sw86
Running command git version
git version 2.34.1
Running command git clone --filter=blob:none -q https://github.com/yaml/pyyaml.git /tmp/pip-req-build-rdv2sw86
Running command git rev-parse HEAD
8cdff2c80573b8be8e8ad28929264a913a63aa33
Resolved https://github.com/yaml/pyyaml.git to commit 8cdff2c80573b8be8e8ad28929264a913a63aa33
Running command /home/florian/tmp/toxtest/.venv/bin/python3 /tmp/pip-standalone-pip-7sp3xayz/__env_pip__.zip/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-az2k859x/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple --pre -- setuptools wheel Cython
Collecting setuptools
Using cached setuptools-60.5.0-py3-none-any.whl (958 kB)
Collecting wheel
Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Collecting Cython
Using cached Cython-3.0.0a10-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (2.2 MB)
Installing collected packages: wheel, setuptools, Cython
Successfully installed Cython-3.0.0a10 setuptools-60.5.0 wheel-0.37.1
Installing build dependencies ... done
Running command /home/florian/tmp/toxtest/.venv/bin/python3 /home/florian/tmp/toxtest/.venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpqirb9t5l
running egg_info
creating lib/PyYAML.egg-info
writing manifest file 'lib/PyYAML.egg-info/SOURCES.txt'
Traceback (most recent call last):
File "/home/florian/tmp/toxtest/.venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
main()
File "/home/florian/tmp/toxtest/.venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/home/florian/tmp/toxtest/.venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 130, in get_requires_for_build_wheel
return hook(config_settings)
File "/tmp/pip-build-env-az2k859x/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 162, in get_requires_for_build_wheel
return self._get_build_requires(
File "/tmp/pip-build-env-az2k859x/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 143, in _get_build_requires
self.run_setup()
File "/tmp/pip-build-env-az2k859x/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 158, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 288, in <module>
setup(
File "/tmp/pip-build-env-az2k859x/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 155, in setup
return distutils.core.setup(**attrs)
File "/tmp/pip-build-env-az2k859x/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 148, in setup
return run_commands(dist)
File "/tmp/pip-build-env-az2k859x/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 163, in run_commands
dist.run_commands()
File "/tmp/pip-build-env-az2k859x/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands
self.run_command(cmd)
File "/tmp/pip-build-env-az2k859x/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
cmd_obj.run()
File "/tmp/pip-build-env-az2k859x/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 299, in run
self.find_sources()
File "/tmp/pip-build-env-az2k859x/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 306, in find_sources
mm.run()
File "/tmp/pip-build-env-az2k859x/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 541, in run
self.add_defaults()
File "/tmp/pip-build-env-az2k859x/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 578, in add_defaults
sdist.add_defaults(self)
File "/tmp/pip-build-env-az2k859x/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/sdist.py", line 228, in add_defaults
self._add_defaults_ext()
File "/tmp/pip-build-env-az2k859x/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/sdist.py", line 312, in _add_defaults_ext
self.filelist.extend(build_ext.get_source_files())
File "setup.py", line 204, in get_source_files
self.cython_sources(ext.sources, ext)
File "/tmp/pip-build-env-az2k859x/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 103, in __getattr__
raise AttributeError(attr)
AttributeError: cython_sources
Getting requirements to build wheel ... error
WARNING: Discarding git+https://github.com/yaml/pyyaml.git. Command errored out with exit status 1: /home/florian/tmp/toxtest/.venv/bin/python3 /home/florian/tmp/toxtest/.venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpqirb9t5l Check the logs for full command output.
ERROR: Command errored out with exit status 1: /home/florian/tmp/toxtest/.venv/bin/python3 /home/florian/tmp/toxtest/.venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpqirb9t5l Check the logs for full command output.
I'm not sure if this is an intended change in Cython 3, or a bug - but given that "pyyaml setup is still relying on a bunch of ancient custom distutils bits" (#584) I thought I'd report it here first. Please let me know if you think I should rather report it there.
It's definitely a regression between Cython 3.0.0a9 and a10; every previous 3.0.0 alpha has been able to build the pyyaml extension fine. This PR (included in a10) swapped out the base class on Cython's custom build_ext distutils command (which pyyaml further subclasses to customize some stuff). I haven't dug down all the way yet to see if it's something we should try to address in pyyaml's customization or if it's just a bug in the new base class.
Thanks for the report- regardless, we'll want to get this sorted one way or another, so I'll figure out where the problem is and work with the Cython/setuptools folks if necessary to get it working.
I encountered this error when attempting to lock my dependencies with pipenv; I was able to fix it in my case by setting
[pipenv]
allow_prereleases = false
just figured I'd share this in case it helps someone else.
allow_prereleases = false
That'll only work until Cython 3.0 releases, unless we're able to get a change made over there (which it sounds like they're amenable to, if it's not too gnarly)
I don't know if this can be of interest for anyone reading here, but I had the same issue in a GitHub Workflow, only happening on Python 3.10. The failed build can be seen here: https://github.com/Neoteroi/BlackSheep/runs/5098546591?check_suite_focus=true
In my case, PyYAML is a dependency and I was testing another wheel with this command:
pip install --pre blacksheep -f "file:///${GITHUB_WORKSPACE}/dist"In my case, removing the --pre option fixed the problem.
pip install blacksheep -f "file:///${GITHUB_WORKSPACE}/dist"Yep, unfortunately --pre seems to affect all deps and build deps as well under pip.
Ping. Cython 3.0.0 has already reached beta releases, so we need some solution sooner than later.
Cool- we've been poking at doing another release, and since we sat on it long enough, setuptools and pip now fully support PEP517/518 config passthru from the CLI, which should make the replacement of the dynamic bits a whole lot easier.
Stumbled upon this bug while trying to install Pycaret:
Collecting PyYAML~=5.1
#0 30.24 Downloading PyYAML-5.4.1.tar.gz (175 kB) #0 30.24 ββββββββββββββββββββββββββββββββββββββ 175.1/175.1 kB 34.3 MB/s eta 0:00:00
#0 30.32 Installing build dependencies: started
#0 33.08 Installing build dependencies: finished with status 'done'
#0 33.08 Getting requirements to build wheel: started
#0 33.19 Getting requirements to build wheel: finished with status 'error' #0 33.20 error: subprocess-exited-with-error
#0 33.20
#0 33.20 Γ Getting requirements to build wheel did not run successfully.
#0 33.20 β exit code: 1
#0 33.20 β°β> [50 lines of output]
#0 33.20 /tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/config/setupcfg.py:516: SetuptoolsDeprecationWarning: T
he license_file parameter is deprecated, use license_files instead.
#0 33.20 warnings.warn(msg, warning_class)
#0 33.20 running egg_info
#0 33.20 writing lib3/PyYAML.egg-info/PKG-INFO
#0 33.20 writing dependency_links to lib3/PyYAML.egg-info/dependency_links.txt
#0 33.20 writing top-level names to lib3/PyYAML.egg-info/top_level.txt
#0 33.20 Traceback (most recent call last):
#0 33.20 File "/opt/conda/envs/rapids/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <
module>
#0 33.20 main()
#0 33.20 File "/opt/conda/envs/rapids/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in m
ain
#0 33.20 json_out['return_val'] = hook(**hook_input['kwargs'])
#0 33.20 File "/opt/conda/envs/rapids/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in g
et_requires_for_build_wheel
#0 33.20 return hook(config_settings)
#0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in get_requires_for_b
uild_wheel
#0 33.20 return self._get_build_requires(config_settings, requirements=['wheel'])
#0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 320, in _get_build_require
s
#0 33.20 self.run_setup()
#0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 335, in run_setup
#0 33.20 exec(code, locals())
#0 33.20 File "<string>", line 271, in <module>
#0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 108, in setup
#0 33.20 return distutils.core.setup(**attrs)
#0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
#0 33.20 return run_commands(dist)
#0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
#0 33.20 dist.run_commands()
#0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
#0 33.20 self.run_command(cmd)
#0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1221, in run_command
#0 33.20 super().run_command(command)
#0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
#0 33.20 cmd_obj.run()
#0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 318, in run
#0 33.20 self.find_sources()
#0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 326, in find_sources
#0 33.20 mm.run()
#0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 570, in run
#0 33.20 self.add_defaults()
#0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 608, in add_defaults
#0 33.20 sdist.add_defaults(self)
#0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/command/sdist.py", line 106, in add_defaults
#0 33.20 super().add_defaults()
#0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
#0 33.20 self._add_defaults_ext()
#0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
#0 33.20 self.filelist.extend(build_ext.get_source_files())
#0 33.20 File "<string>", line 201, in get_source_files
#0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
#0 33.20 raise AttributeError(attr)
#0 33.20 AttributeError: cython_sources
#0 33.20 [end of output]@jmakov the likely short-term fix will be to pin the Cython PEP518 build deps to pre-3.0 Cython in a 6.x release (see #702), but we generally consider the 5.x release branch dead, so hadn't planned on re-publishing the build change back there. You might want to try and get the Pycaret folks (or whoever's ultimately pinning PyYAML to ~=5.1) to update their deps.
@nitzmahone thank you so much for pointing out the solution! Will link your answer to an issue in their repo.
@jmakov after looking at the issue you just linked, now I think I see how you ended up building PyYAML with an unreleased Cython 3.x; IIRC pip's --pre option globally allows pre-releases to be considered for installation (so for the package you asked for, as well as its entire dep tree and any build requirements pip installs).
If you just ask pip for for a specific pre-release version of Pycaret (without using the --pre flag, eg pip install pycaret==3.0.0rc8), the normal runtime and build-dep rules will be applied and PyYAML shouldn't try to build with an unreleased Cython.
@nitzmahone that's a very interesting gotcha! This also explains why this works, thanks!
This is now even more urgent as Cython<3 doesn't support Python 3.12.
@mgorny where are you hearing that? Empirically speaking, Cython 0.29.x has been working flawlessly on 3.12- PyYAML extension builds have been passing the full test suite all along through the current Python 3.12.0b1.
(not that I don't want a better excuse to spend the time shredding PyYAML's creaky old packaging code to work under Cython3 while still doing everything it needs to do π )
@nitzmahone That's my bad, I think! The tests failed immediately (try to import imp, keeping an eye on cython/cython#5285) but I thought it'd failed to compile too.
I'm going to test a bunch of reverse dependencies like pyyaml in Gentoo and unleash it if it works. Apologies for the error!
EDIT: Done!
Any update on this, now that Cython 3.0 has been released?
As with all the mentions, this is breaking our ability to build our production apps.
Is there any chance of getting a release that pins Cython? Failing that does anyone know how to work around this by pinning Cython when using poetry?
Edit: I've realized that this is happening for PyYaml 6.0 too
Our workaround for the time being is, in our build process, to install a compatible cython first and then use --no-build-isolation. Eg
pip install "Cython<3.0" pyyaml --no-build-isolation # new line
pip install -r requirements.txt # original build step, which has pyyaml in it
if that helps anyone
This started effecting our builds this morning. In our case we had pinned pyyaml to 5.x, so I just had to upgrade to 6.x and things seem to be working.
Just building on @pinghedm's solution. If you happen to have another package that depends on pyyaml 5.x (like awscli). You can install it like this:
pip install "Cython<3.0" "pyyaml<6" --no-build-isolation
pip install -r requirements.txt
@MisterGlass Are you using Alpine, by any chance?
^^above not working for me
Here's my dockerFile
FROM python:3.10-alpine
RUN pip install "Cython<3.0" pyyaml --no-build-isolation
RUN pip install -r requirements.txt
- requirements.txt
PyGithub==1.57
PyYAML==6.0
we're already on pyyaml 6.0 but are affected by this issue. we're on python:3.9.16-alpine3.17 and @pinghedm's workaround unblocks us.
this is the error that I get after trying @pinghedm 's workaround
.80 Collecting pycparser
18.82 Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
19.77 βββββββββββββββββββββββββββββββββββββ 118.7/118.7 kB 122.1 kB/s eta 0:00:00
19.79 Building wheels for collected packages: cffi
19.79 Building wheel for cffi (setup.py): started
20.08 Building wheel for cffi (setup.py): finished with status 'error'
20.09 error: subprocess-exited-with-error
20.09
20.09 Γ python setup.py bdist_wheel did not run successfully.
20.09 β exit code: 1
20.09 β°β> [47 lines of output]
20.09
20.09 No working compiler found, or bogus compiler options passed to
20.09 the compiler from Python's standard "distutils" module. See
20.09 the error messages above. Likely, the problem is not related
20.09 to CFFI but generic to the setup.py of any Python package that
20.09 tries to compile C code. (Hints: on OS/X 10.8, for errors about
20.09 -mno-fused-madd see http://stackoverflow.com/questions/22313407/
20.09 Otherwise, see https://wiki.python.org/moin/CompLangPython or
20.09 the IRC channel #python on irc.libera.chat.)
20.09
20.09 Trying to continue anyway. If you are trying to install CFFI from
20.09 a build done in a different context, you can ignore this warning.
20.09
20.09 /usr/local/lib/python3.10/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
20.09 warnings.warn(msg, warning_class)
20.09 running bdist_wheel
20.09 running build
20.09 running build_py
20.09 creating build
20.09 creating build/lib.linux-aarch64-cpython-310
20.09 creating build/lib.linux-aarch64-cpython-310/cffi
20.09 copying cffi/setuptools_ext.py -> build/lib.linux-aarch64-cpython-310/cffi
20.09 copying cffi/__init__.py -> build/lib.linux-aarch64-cpython-310/cffi
20.09 copying cffi/vengine_cpy.py -> build/lib.linux-aarch64-cpython-310/cffi
20.09 copying cffi/verifier.py -> build/lib.linux-aarch64-cpython-310/cffi
20.09 copying cffi/ffiplatform.py -> build/lib.linux-aarch64-cpython-310/cffi
20.09 copying cffi/cffi_opcode.py -> build/lib.linux-aarch64-cpython-310/cffi
20.09 copying cffi/recompiler.py -> build/lib.linux-aarch64-cpython-310/cffi
20.09 copying cffi/error.py -> build/lib.linux-aarch64-cpython-310/cffi
20.09 copying cffi/commontypes.py -> build/lib.linux-aarch64-cpython-310/cffi
20.09 copying cffi/backend_ctypes.py -> build/lib.linux-aarch64-cpython-310/cffi
20.09 copying cffi/lock.py -> build/lib.linux-aarch64-cpython-310/cffi
20.09 copying cffi/api.py -> build/lib.linux-aarch64-cpython-310/cffi
20.09 copying cffi/cparser.py -> build/lib.linux-aarch64-cpython-310/cffi
20.09 copying cffi/pkgconfig.py -> build/lib.linux-aarch64-cpython-310/cffi
20.09 copying cffi/model.py -> build/lib.linux-aarch64-cpython-310/cffi
20.09 copying cffi/vengine_gen.py -> build/lib.linux-aarch64-cpython-310/cffi
20.09 copying cffi/_cffi_include.h -> build/lib.linux-aarch64-cpython-310/cffi
20.09 copying cffi/parse_c_type.h -> build/lib.linux-aarch64-cpython-310/cffi
20.09 copying cffi/_embedding.h -> build/lib.linux-aarch64-cpython-310/cffi
20.09 copying cffi/_cffi_errors.h -> build/lib.linux-aarch64-cpython-310/cffi
20.09 running build_ext
20.09 building '_cffi_backend' extension
20.09 creating build/temp.linux-aarch64-cpython-310
20.09 creating build/temp.linux-aarch64-cpython-310/c
20.09 gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DFFI_BUILDING=1 -I/usr/include/ffi -I/usr/include/libffi -I/usr/local/include/python3.10 -c c/_cffi_backend.c -o build/temp.linux-aarch64-cpython-310/c/_cffi_backend.o
20.09 error: command 'gcc' failed: No such file or directory
20.09 [end of output]
20.09
20.09 note: This error originates from a subprocess, and is likely not a problem with pip.
20.09 ERROR: Failed building wheel for cffi
20.09 Running setup.py clean for cffi
20.21 Failed to build cffi
20.25 Installing collected packages: wrapt, urllib3, pyjwt, pycparser, idna, charset-normalizer, certifi, requests, deprecated, cffi, pynacl, PyGithub
20.54 Running setup.py install for cffi: started
20.69 Running setup.py install for cffi: finished with status 'error'
20.70 error: subprocess-exited-with-error
20.70
20.70 Γ Running setup.py install for cffi did not run successfully.
20.70 β exit code: 1
20.70 β°β> [49 lines of output]
20.70
20.70 No working compiler found, or bogus compiler options passed to
20.70 the compiler from Python's standard "distutils" module. See
20.70 the error messages above. Likely, the problem is not related
20.70 to CFFI but generic to the setup.py of any Python package that
20.70 tries to compile C code. (Hints: on OS/X 10.8, for errors about
20.70 -mno-fused-madd see http://stackoverflow.com/questions/22313407/
20.70 Otherwise, see https://wiki.python.org/moin/CompLangPython or
20.70 the IRC channel #python on irc.libera.chat.)
20.70
20.70 Trying to continue anyway. If you are trying to install CFFI from
20.70 a build done in a different context, you can ignore this warning.
20.70
20.70 /usr/local/lib/python3.10/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
20.70 warnings.warn(msg, warning_class)
20.70 running install
20.70 /usr/local/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
20.70 warnings.warn(
20.70 running build
20.70 running build_py
20.70 creating build
20.70 creating build/lib.linux-aarch64-cpython-310
20.70 creating build/lib.linux-aarch64-cpython-310/cffi
20.70 copying cffi/setuptools_ext.py -> build/lib.linux-aarch64-cpython-310/cffi
20.70 copying cffi/__init__.py -> build/lib.linux-aarch64-cpython-310/cffi
20.70 copying cffi/vengine_cpy.py -> build/lib.linux-aarch64-cpython-310/cffi
20.70 copying cffi/verifier.py -> build/lib.linux-aarch64-cpython-310/cffi
20.70 copying cffi/ffiplatform.py -> build/lib.linux-aarch64-cpython-310/cffi
20.70 copying cffi/cffi_opcode.py -> build/lib.linux-aarch64-cpython-310/cffi
20.70 copying cffi/recompiler.py -> build/lib.linux-aarch64-cpython-310/cffi
20.70 copying cffi/error.py -> build/lib.linux-aarch64-cpython-310/cffi
20.70 copying cffi/commontypes.py -> build/lib.linux-aarch64-cpython-310/cffi
20.70 copying cffi/backend_ctypes.py -> build/lib.linux-aarch64-cpython-310/cffi
20.70 copying cffi/lock.py -> build/lib.linux-aarch64-cpython-310/cffi
20.70 copying cffi/api.py -> build/lib.linux-aarch64-cpython-310/cffi
20.70 copying cffi/cparser.py -> build/lib.linux-aarch64-cpython-310/cffi
20.70 copying cffi/pkgconfig.py -> build/lib.linux-aarch64-cpython-310/cffi
20.70 copying cffi/model.py -> build/lib.linux-aarch64-cpython-310/cffi
20.70 copying cffi/vengine_gen.py -> build/lib.linux-aarch64-cpython-310/cffi
20.70 copying cffi/_cffi_include.h -> build/lib.linux-aarch64-cpython-310/cffi
20.70 copying cffi/parse_c_type.h -> build/lib.linux-aarch64-cpython-310/cffi
20.70 copying cffi/_embedding.h -> build/lib.linux-aarch64-cpython-310/cffi
20.70 copying cffi/_cffi_errors.h -> build/lib.linux-aarch64-cpython-310/cffi
20.70 running build_ext
20.70 building '_cffi_backend' extension
20.70 creating build/temp.linux-aarch64-cpython-310
20.70 creating build/temp.linux-aarch64-cpython-310/c
20.70 gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DFFI_BUILDING=1 -I/usr/include/ffi -I/usr/include/libffi -I/usr/local/include/python3.10 -c c/_cffi_backend.c -o build/temp.linux-aarch64-cpython-310/c/_cffi_backend.o
20.70 error: command 'gcc' failed: No such file or directory
20.70 [end of output]
20.70
20.70 note: This error originates from a subprocess, and is likely not a problem with pip.
20.70 error: legacy-install-failure
20.70
20.70 Γ Encountered error while trying to install package.
20.70 β°β> cffi
20.70
20.70 note: This is an issue with the package mentioned above, not pip.
20.70 hint: See above for output from the failure.
20.70
20.70 [notice] A new release of pip is available: 23.0.1 -> 23.2
20.70 [notice] To update, run: pip install --upgrade pip
------
Dockerfile:7
--------------------
5 | # Install dependencies
6 | RUN pip install "Cython<3.0" pyyaml --no-build-isolation
7 | >>> RUN pip install -r requirements.txt
8 |
9 | # Set the entrypoint
--------------------
ERROR: failed to solve: process "/bin/sh -c pip install -r requirements.txt" did not complete successfully: exit code: 1
Same as @muleyashutosh -- the complete Dockerfile is:
FROM python:alpine
RUN apk add bash
RUN pip install "Cython<3.0" "pyyaml<6" --no-build-isolation
RUN pip wheel install --no-cache-dir cfn-lint==0.78.1
error: command 'gcc' failed: No such file or directory
Install gcc and other required build dependencies before doing the pip install (apk add gcc etc)
Workaound doesn't work for me as well (no gcc error though)
FROM python:3.11.4-alpine3.18
RUN apk add \
bash \
conntrack-tools \
iptables \
&& rm -rf /var/cache/apk/*
WORKDIR /usr/src/kamailio-metrics/
COPY requirements.txt .
COPY pystn ./pystn
RUN python3 -m venv .venv && \
source .venv/bin/activate && \
pip install --upgrade pip && \
pip install "Cython<3.0" "pyyaml<6" --no-build-isolation && \
pip wheel install --no-cache-dir cfn-lint==0.78.1
β― make build && make push
docker build -t jcr.io/platform/kamailio-metrics:jdesmarais .
[+] Building 10.0s (10/11) docker:desktop-linux
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.06kB 0.0s
=> [internal] load metadata for docker.io/library/python:3.11.4-alpine3.18 0.8s
=> [1/7] FROM docker.io/library/python:3.11.4-alpine3.18@sha256:25df32b602118dab046b58f0fe920e3301da0727b5b07430c8bcd4b139627fdc 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 3.78kB 0.0s
=> CACHED [2/7] RUN apk add bash conntrack-tools iptables && rm -rf /var/cache/apk/* 0.0s
=> CACHED [3/7] WORKDIR /usr/src/kamailio-metrics/ 0.0s
=> CACHED [4/7] COPY requirements.txt . 0.0s
=> CACHED [5/7] COPY pystn ./pystn 0.0s
=> ERROR [6/7] RUN python3 -m venv .venv && source .venv/bin/activate && pip install --upgrade pip && pip install "Cython<3.0" "pyyaml<6" --no-build-isolation && pip wheel install --no-cache-dir cfn-lint==0.78.1 9.1s
------
> [6/7] RUN python3 -m venv .venv && source .venv/bin/activate && pip install --upgrade pip && pip install "Cython<3.0" "pyyaml<6" --no-build-isolation && pip wheel install --no-cache-dir cfn-lint==0.78.1 && pip install -r requirements.txt:
4.962 Requirement already satisfied: pip in ./.venv/lib/python3.11/site-packages (23.1.2)
5.115 Collecting pip
5.379 Downloading pip-23.2-py3-none-any.whl (2.1 MB)
5.611 ββββββββββββββββββββββββββββββββββββββββ 2.1/2.1 MB 9.1 MB/s eta 0:00:00
5.676 Installing collected packages: pip
5.676 Attempting uninstall: pip
5.679 Found existing installation: pip 23.1.2
5.727 Uninstalling pip-23.1.2:
5.734 Successfully uninstalled pip-23.1.2
7.056 Successfully installed pip-23.2
8.028 Collecting Cython<3.0
8.028 Obtaining dependency information for Cython<3.0 from https://files.pythonhosted.org/packages/cf/a3/43bf5b3ba528a8fa2aa25dd48317e493b46ce838098605277b9efc7a0c1a/Cython-0.29.36-cp311-cp311-musllinux_1_1_x86_64.whl.metadata
8.264 Downloading Cython-0.29.36-cp311-cp311-musllinux_1_1_x86_64.whl.metadata (3.1 kB)
8.333 Collecting pyyaml<6
8.377 Downloading PyYAML-5.4.1.tar.gz (175 kB)
8.479 βββββββββββββββββββββββββββββββββββββββ 175.1/175.1 kB 1.8 MB/s eta 0:00:00
8.651 Preparing metadata (pyproject.toml): started
8.929 Preparing metadata (pyproject.toml): finished with status 'error'
8.935 error: subprocess-exited-with-error
8.935
8.935 Γ Preparing metadata (pyproject.toml) did not run successfully.
8.935 β exit code: 1
8.935 β°β> [14 lines of output]
8.935 /usr/src/kamailio-metrics/.venv/lib/python3.11/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
8.935 warnings.warn(msg, warning_class)
8.935 running dist_info
8.935 creating /tmp/pip-modern-metadata-i70halss/PyYAML.egg-info
8.935 writing /tmp/pip-modern-metadata-i70halss/PyYAML.egg-info/PKG-INFO
8.935 writing dependency_links to /tmp/pip-modern-metadata-i70halss/PyYAML.egg-info/dependency_links.txt
8.935 writing top-level names to /tmp/pip-modern-metadata-i70halss/PyYAML.egg-info/top_level.txt
8.935 writing manifest file '/tmp/pip-modern-metadata-i70halss/PyYAML.egg-info/SOURCES.txt'
8.935 reading manifest file '/tmp/pip-modern-metadata-i70halss/PyYAML.egg-info/SOURCES.txt'
8.935 reading manifest template 'MANIFEST.in'
8.935 adding license file 'LICENSE'
8.935 writing manifest file '/tmp/pip-modern-metadata-i70halss/PyYAML.egg-info/SOURCES.txt'
8.935 creating '/tmp/pip-modern-metadata-i70halss/PyYAML-5.4.1.dist-info'
8.935 error: invalid command 'bdist_wheel'
8.935 [end of output]
8.935
8.935 note: This error originates from a subprocess, and is likely not a problem with pip.
8.938 error: metadata-generation-failed
8.938
8.938 Γ Encountered error while generating package metadata.
8.938 β°β> See above for output.
8.938
8.938 note: This is an issue with the package mentioned above, not pip.
8.938 hint: See above for details.
------
Dockerfile:38
--------------------
37 |
38 | >>> RUN python3 -m venv .venv && \
39 | >>> source .venv/bin/activate && \
40 | >>> pip install --upgrade pip && \
41 | >>> pip install "Cython<3.0" "pyyaml<6" --no-build-isolation && \
42 | >>> pip wheel install --no-cache-dir cfn-lint==0.78.1 && \
43 | >>> pip install -r requirements.txt
44 |
--------------------
ERROR: failed to solve: process "/bin/sh -c python3 -m venv .venv && source .venv/bin/activate && pip install --upgrade pip && pip install \"Cython<3.0\" \"pyyaml<6\" --no-build-isolation && pip wheel install --no-cache-dir cfn-lint==0.78.1 && pip install -r requirements.txt" did not complete successfully: exit code: 1
@MisterGlass Are you using Alpine, by any chance?
No, I'm using python:3.11 as a base
@jimbojd72 I had the same problem and the resolution was to pip install wheel before the pip install "Cython<3.0" ... command
For folks relying on aws-cli, pinning pyyaml<=5.3.1 helped us.
NOTE - please read this note from aws-cli maintainers before you use pyyaml<=5.3.1:
We do not recommend installing an older version of PyYAML as PyYAML version 5.3.1 is associated with CVE-2020-14343 that was fixed in version 5.4.
For folks relying on aws-cli, pinning
pyyaml<=5.3.1helped us.
Thanks @romilbhardwaj that did the trick for us too!
adding pip install wheel fixed it for us
FROM python:alpine
RUN apk add bash
RUN pip install wheel
RUN pip install "Cython<3.0" "pyyaml<6" --no-build-isolation
RUN pip install --no-cache-dir cfn-lint==0.65.1
this fixed it for me.
RUN apk update && apk add build-base libffi-dev
RUN pip install "Cython<3.0" pyyaml --no-build-isolation
RUN pip install -r requirements.txt
Reporting identical issues in an Ubuntu 20.04 container running Python 3.10.1 compiled from source + pip 21.3.
cython compilation error specifically with pyyaml==5.4.1, installs fine with latest (6.0) or 5.3.1.
Sad morning to discover PEP 518 (--no-build-isolation). So much for freezing my requirements.txt π
@nitzmahone is there anyway we can get #702 merged and released in 6.x to help stem the current build issues? There unfortunately isn't a robust way for downstream packages to mitigate this without defining a runtime dependency on Cython for constraint.
Is there any way to specify --no-build-isolation inside of a setup.cfg or requirements.txt file?
Workaound doesn't work for me as well (no gcc error though)
FROM python:3.11.4-alpine3.18 RUN apk add \ bash \ conntrack-tools \ iptables \ && rm -rf /var/cache/apk/* WORKDIR /usr/src/kamailio-metrics/ COPY requirements.txt . COPY pystn ./pystn RUN python3 -m venv .venv && \ source .venv/bin/activate && \ pip install --upgrade pip && \ pip install "Cython<3.0" "pyyaml<6" --no-build-isolation && \ pip wheel install --no-cache-dir cfn-lint==0.78.1β― make build && make push docker build -t jcr.io/platform/kamailio-metrics:jdesmarais . [+] Building 10.0s (10/11) docker:desktop-linux => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 1.06kB 0.0s => [internal] load metadata for docker.io/library/python:3.11.4-alpine3.18 0.8s => [1/7] FROM docker.io/library/python:3.11.4-alpine3.18@sha256:25df32b602118dab046b58f0fe920e3301da0727b5b07430c8bcd4b139627fdc 0.0s => [internal] load build context 0.0s => => transferring context: 3.78kB 0.0s => CACHED [2/7] RUN apk add bash conntrack-tools iptables && rm -rf /var/cache/apk/* 0.0s => CACHED [3/7] WORKDIR /usr/src/kamailio-metrics/ 0.0s => CACHED [4/7] COPY requirements.txt . 0.0s => CACHED [5/7] COPY pystn ./pystn 0.0s => ERROR [6/7] RUN python3 -m venv .venv && source .venv/bin/activate && pip install --upgrade pip && pip install "Cython<3.0" "pyyaml<6" --no-build-isolation && pip wheel install --no-cache-dir cfn-lint==0.78.1 9.1s ------ > [6/7] RUN python3 -m venv .venv && source .venv/bin/activate && pip install --upgrade pip && pip install "Cython<3.0" "pyyaml<6" --no-build-isolation && pip wheel install --no-cache-dir cfn-lint==0.78.1 && pip install -r requirements.txt: 4.962 Requirement already satisfied: pip in ./.venv/lib/python3.11/site-packages (23.1.2) 5.115 Collecting pip 5.379 Downloading pip-23.2-py3-none-any.whl (2.1 MB) 5.611 ββββββββββββββββββββββββββββββββββββββββ 2.1/2.1 MB 9.1 MB/s eta 0:00:00 5.676 Installing collected packages: pip 5.676 Attempting uninstall: pip 5.679 Found existing installation: pip 23.1.2 5.727 Uninstalling pip-23.1.2: 5.734 Successfully uninstalled pip-23.1.2 7.056 Successfully installed pip-23.2 8.028 Collecting Cython<3.0 8.028 Obtaining dependency information for Cython<3.0 from https://files.pythonhosted.org/packages/cf/a3/43bf5b3ba528a8fa2aa25dd48317e493b46ce838098605277b9efc7a0c1a/Cython-0.29.36-cp311-cp311-musllinux_1_1_x86_64.whl.metadata 8.264 Downloading Cython-0.29.36-cp311-cp311-musllinux_1_1_x86_64.whl.metadata (3.1 kB) 8.333 Collecting pyyaml<6 8.377 Downloading PyYAML-5.4.1.tar.gz (175 kB) 8.479 βββββββββββββββββββββββββββββββββββββββ 175.1/175.1 kB 1.8 MB/s eta 0:00:00 8.651 Preparing metadata (pyproject.toml): started 8.929 Preparing metadata (pyproject.toml): finished with status 'error' 8.935 error: subprocess-exited-with-error 8.935 8.935 Γ Preparing metadata (pyproject.toml) did not run successfully. 8.935 β exit code: 1 8.935 β°β> [14 lines of output] 8.935 /usr/src/kamailio-metrics/.venv/lib/python3.11/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead. 8.935 warnings.warn(msg, warning_class) 8.935 running dist_info 8.935 creating /tmp/pip-modern-metadata-i70halss/PyYAML.egg-info 8.935 writing /tmp/pip-modern-metadata-i70halss/PyYAML.egg-info/PKG-INFO 8.935 writing dependency_links to /tmp/pip-modern-metadata-i70halss/PyYAML.egg-info/dependency_links.txt 8.935 writing top-level names to /tmp/pip-modern-metadata-i70halss/PyYAML.egg-info/top_level.txt 8.935 writing manifest file '/tmp/pip-modern-metadata-i70halss/PyYAML.egg-info/SOURCES.txt' 8.935 reading manifest file '/tmp/pip-modern-metadata-i70halss/PyYAML.egg-info/SOURCES.txt' 8.935 reading manifest template 'MANIFEST.in' 8.935 adding license file 'LICENSE' 8.935 writing manifest file '/tmp/pip-modern-metadata-i70halss/PyYAML.egg-info/SOURCES.txt' 8.935 creating '/tmp/pip-modern-metadata-i70halss/PyYAML-5.4.1.dist-info' 8.935 error: invalid command 'bdist_wheel' 8.935 [end of output] 8.935 8.935 note: This error originates from a subprocess, and is likely not a problem with pip. 8.938 error: metadata-generation-failed 8.938 8.938 Γ Encountered error while generating package metadata. 8.938 β°β> See above for output. 8.938 8.938 note: This is an issue with the package mentioned above, not pip. 8.938 hint: See above for details. ------ Dockerfile:38 -------------------- 37 | 38 | >>> RUN python3 -m venv .venv && \ 39 | >>> source .venv/bin/activate && \ 40 | >>> pip install --upgrade pip && \ 41 | >>> pip install "Cython<3.0" "pyyaml<6" --no-build-isolation && \ 42 | >>> pip wheel install --no-cache-dir cfn-lint==0.78.1 && \ 43 | >>> pip install -r requirements.txt 44 | -------------------- ERROR: failed to solve: process "/bin/sh -c python3 -m venv .venv && source .venv/bin/activate && pip install --upgrade pip && pip install \"Cython<3.0\" \"pyyaml<6\" --no-build-isolation && pip wheel install --no-cache-dir cfn-lint==0.78.1 && pip install -r requirements.txt" did not complete successfully: exit code: 1
Building with pyyaml pinned to version 5.3.1 fixed this issue for me
RUN python3 -m venv .venv && \
source .venv/bin/activate && \
pip install --upgrade pip && \
pip install 'pyyaml<=5.3.1' && \
pip install wheel && \
pip install -r requirements.txt
nitzmahone is there anyway we can get #702 merged and released in 6.x to help stem the current build issues? There unfortunately isn't a robust way for downstream packages to mitigate this without defining a runtime dependency on Cython for constraint.
Are you running into this issue with 6.0?
5.4.1 blows up for me, but a fresh virtualenv has no problem with pip install pyyaml==6.0
There unfortunately isn't a robust way for downstream packages to mitigate this without defining a runtime dependency on Cython for constraint.
For some package managers, e.g. Poetry, even defining a Cython runtime dependency doesn't help because Poetry builds wheels for sdist packages upon installation in a temporary directory that's isolated from the runtime environment and only includes packages defined in PEP 517's build-requires
The specific issue here is if you're trying to install PyYAML on a system that doesn't have a wheel available. Cython is only involved when building from source. Most of the report are on Alpine because there aren't musl wheels available. If 6.x is working, my guess is you're using Python 3.10 or 3.11 which didn't have wheels available in the 5.4.1 release.
This will continue to be an issue until the build dependencies are properly constrained.
Anyone using poetry? It seems I only can use 6 version with it.
Does anyone had the same problem?
The specific issue here is if you're trying to install PyYAML on a system that doesn't have a wheel available. Cython is only involved when building from source. Most of the report are on Alpine because there aren't musl wheels available. If 6.x is working, my guess is you're using Python 3.10 or 3.11 which didn't have wheels available in the 5.4.1 release.
This will continue to be an issue until the build dependencies are properly constrained.
I'm using python 11 on alpine on a Fresh environment using Poetry... But, as mentioned by kylebarron, this affects me anyways...
Most of the report are on Alpine because there aren't musl wheels available.
All Alpine images are affected.
This is breaking all my testing for 3.12, which doesn't have wheels (they can be uploaded pretty safely now with 3.12.0b4).
You can set PIP_CONSTRAINTS_FILE to a file with a constraint on pyyaml that will lock an older version if that helps. Or on Cython, which would help, I think. I'm not sure Poetry will respect it, but most tools use pip which will.
For my poetry based projects building in docker I was able to work around this by doing the following in my docker file:
RUN poetry run pip install "Cython<3.0" pyyaml --no-build-isolation && \
poetry install@Kaelten thanks for the suggestion! That works for me on Poetry 1.5.1!
@r-asiebert
Are you running into this issue with 6.0?
It does, at least for me (I cut in the stack trace for brevity):
$ docker run --rm python:alpine python -m pip install PyYAML
Collecting PyYAML
Downloading PyYAML-6.0.tar.gz (124 kB)
βββββββββββββββββββββββββββββββββββββββ 125.0/125.0 kB 4.4 MB/s eta 0:00:00
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'error'
error: subprocess-exited-with-error
Γ Getting requirements to build wheel did not run successfully.
β exit code: 1
β°β> [54 lines of output]
running egg_info
writing lib/PyYAML.egg-info/PKG-INFO
writing dependency_links to lib/PyYAML.egg-info/dependency_links.txt
writing top-level names to lib/PyYAML.egg-info/top_level.txt
Traceback (most recent call last):
(...)
File "/tmp/pip-build-env-n1npha3u/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
raise AttributeError(attr)
AttributeError: cython_sources
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
Γ Getting requirements to build wheel did not run successfully.
β exit code: 1
β°β> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.In poetry, we were able to fix this by updating pyyaml to 6.0.
poetry add pyyaml@latest
That seemed to do the trick.
I would dearly love to see #702 get merged and released as 6.0.1, as a workaround to this issue, at least for folks using version 6.0.0
Does anybody solved the issue for python:3.9-slim-bullseye docker image?
Just building on @pinghedm's solution. If you happen to have another package that depends on pyyaml 5.x (like awscli). You can install it like this:
pip install "Cython<3.0" "pyyaml<6" --no-build-isolation pip install -r requirements.txt
A version of this that worked with us using pipenv install in our CI pipeline:
pipenv run -- pip install "Cython<3.0" "pyyaml<6" --no-build-isolation # new line for workaround
pipenv install # existing line, installs successfully from Pipfile.lock
Could we just get a .post1 out that addresses only this?
Our project's requirements/CI/CD setup is quite complex and it's grounded until this is addressed.
Upgrading to PyYAML>=6.0 worked for most of my projects (Ubuntu 22.04, Python 3.10.8).
However, one project also needs aws-cli which imposes a cap of PyYAML<=5.5. I might have to downgrade Cython in this case, as aws-cli has had an issue open for two years to bump it's PyYAML version.
same for me on python3.9 (alpine3.18 )
#11 17.85 Collecting PyYAML>=5.1 (from drf-spectacular==0.17.3->-r requirements/production.txt (line 15))
#11 17.86 Downloading PyYAML-6.0.tar.gz (124 kB)
#11 17.86 ββββββββββββββββββββββββββββββββββββββ 125.0/125.0 kB 36.9 MB/s eta 0:00:00
#11 17.93 Installing build dependencies: started
#11 20.94 Installing build dependencies: finished with status 'done'
#11 20.94 Getting requirements to build wheel: started
#11 21.05 Getting requirements to build wheel: finished with status 'error'
#11 21.08 error: subprocess-exited-with-error
#11 21.08
#11 21.08 Γ Getting requirements to build wheel did not run successfully.
#11 21.08 β exit code: 1
#11 21.08 β°β> [48 lines of output]
#11 21.08 running egg_info
#11 21.08 writing lib/PyYAML.egg-info/PKG-INFO
#11 21.08 writing dependency_links to lib/PyYAML.egg-info/dependency_links.txt
#11 21.08 writing top-level names to lib/PyYAML.egg-info/top_level.txt
#11 21.08 Traceback (most recent call last):
#11 21.08 File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
#11 21.08 main()
#11 21.08 File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
#11 21.08 json_out['return_val'] = hook(**hook_input['kwargs'])
#11 21.08 File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
#11 21.08 return hook(config_settings)
#11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
#11 21.08 return self._get_build_requires(config_settings, requirements=['wheel'])
#11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
#11 21.08 self.run_setup()
#11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 338, in run_setup
#11 21.08 exec(code, locals())
#11 21.08 File "<string>", line 288, in <module>
#11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 107, in setup
#11 21.08 return distutils.core.setup(**attrs)
#11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 185, in setup
#11 21.08 return run_commands(dist)
#11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
#11 21.08 dist.run_commands()
#11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
#11 21.08 self.run_command(cmd)
#11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
#11 21.08 super().run_command(command)
#11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
#11 21.08 cmd_obj.run()
#11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 314, in run
#11 21.08 self.find_sources()
#11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 322, in find_sources
#11 21.08 mm.run()
#11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 551, in run
#11 21.08 self.add_defaults()
#11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 589, in add_defaults
#11 21.08 sdist.add_defaults(self)
#11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/command/sdist.py", line 104, in add_defaults
#11 21.08 super().add_defaults()
#11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
#11 21.08 self._add_defaults_ext()
#11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
#11 21.08 self.filelist.extend(build_ext.get_source_files())
#11 21.08 File "<string>", line 204, in get_source_files
#11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
#11 21.08 raise AttributeError(attr)
#11 21.08 AttributeError: cython_sources
#11 21.08 [end of output]
#11 21.08
#11 21.08 note: This error originates from a subprocess, and is likely not a problem with pip.
#11 21.08 error: subprocess-exited-with-error
same for me on python3.9 (alpine3.18 )
#11 17.85 Collecting PyYAML>=5.1 (from drf-spectacular==0.17.3->-r requirements/production.txt (line 15)) #11 17.86 Downloading PyYAML-6.0.tar.gz (124 kB) #11 17.86 ββββββββββββββββββββββββββββββββββββββ 125.0/125.0 kB 36.9 MB/s eta 0:00:00 #11 17.93 Installing build dependencies: started #11 20.94 Installing build dependencies: finished with status 'done' #11 20.94 Getting requirements to build wheel: started #11 21.05 Getting requirements to build wheel: finished with status 'error' #11 21.08 error: subprocess-exited-with-error #11 21.08 #11 21.08 Γ Getting requirements to build wheel did not run successfully. #11 21.08 β exit code: 1 #11 21.08 β°β> [48 lines of output] #11 21.08 running egg_info #11 21.08 writing lib/PyYAML.egg-info/PKG-INFO #11 21.08 writing dependency_links to lib/PyYAML.egg-info/dependency_links.txt #11 21.08 writing top-level names to lib/PyYAML.egg-info/top_level.txt #11 21.08 Traceback (most recent call last): #11 21.08 File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module> #11 21.08 main() #11 21.08 File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main #11 21.08 json_out['return_val'] = hook(**hook_input['kwargs']) #11 21.08 File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel #11 21.08 return hook(config_settings) #11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel #11 21.08 return self._get_build_requires(config_settings, requirements=['wheel']) #11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires #11 21.08 self.run_setup() #11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 338, in run_setup #11 21.08 exec(code, locals()) #11 21.08 File "<string>", line 288, in <module> #11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 107, in setup #11 21.08 return distutils.core.setup(**attrs) #11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 185, in setup #11 21.08 return run_commands(dist) #11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 201, in run_commands #11 21.08 dist.run_commands() #11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands #11 21.08 self.run_command(cmd) #11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command #11 21.08 super().run_command(command) #11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command #11 21.08 cmd_obj.run() #11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 314, in run #11 21.08 self.find_sources() #11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 322, in find_sources #11 21.08 mm.run() #11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 551, in run #11 21.08 self.add_defaults() #11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 589, in add_defaults #11 21.08 sdist.add_defaults(self) #11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/command/sdist.py", line 104, in add_defaults #11 21.08 super().add_defaults() #11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults #11 21.08 self._add_defaults_ext() #11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext #11 21.08 self.filelist.extend(build_ext.get_source_files()) #11 21.08 File "<string>", line 204, in get_source_files #11 21.08 File "/tmp/pip-build-env-wdo84unn/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__ #11 21.08 raise AttributeError(attr) #11 21.08 AttributeError: cython_sources #11 21.08 [end of output] #11 21.08 #11 21.08 note: This error originates from a subprocess, and is likely not a problem with pip. #11 21.08 error: subprocess-exited-with-error
I was able to work around this in Alpine. We're using Alpine 3.15, but I think the same solution should work:
Cray-HPE/cray-product-catalog#263
Same for me
$ pip install awscli
Collecting awscli
Downloading awscli-1.29.3-py3-none-any.whl (4.1 MB)
ββββββββββββββββββββββββββββββββββββββββ 4.1/4.1 MB 39.5 MB/s eta 0:00:00
Collecting botocore==1.31.3 (from awscli)
Downloading botocore-1.31.3-py3-none-any.whl (11.0 MB)
βββββββββββββββββββββββββββββββββββββββ 11.0/11.0 MB 116.6 MB/s eta 0:00:00
Collecting docutils<0.17,>=0.10 (from awscli)
Downloading docutils-0.16-py2.py3-none-any.whl (548 kB)
ββββββββββββββββββββββββββββββββββββββ 548.2/548.2 kB 57.2 MB/s eta 0:00:00
Collecting s3transfer<0.7.0,>=0.6.0 (from awscli)
Downloading s3transfer-0.6.1-py3-none-any.whl (79 kB)
ββββββββββββββββββββββββββββββββββββββββ 79.8/79.8 kB 13.4 MB/s eta 0:00:00
Collecting PyYAML<5.5,>=3.10 (from awscli)
Downloading PyYAML-5.4.1.tar.gz (175 kB)
ββββββββββββββββββββββββββββββββββββββ 175.1/175.1 kB 27.3 MB/s eta 0:00:00
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'error'
error: subprocess-exited-with-error
Γ Getting requirements to build wheel did not run successfully.
β exit code: 1
β°β> [68 lines of output]
/tmp/pip-build-env-rlbi9ugx/overlay/lib/python3.11/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`
!!
********************************************************************************
The license_file parameter is deprecated, use license_files instead.
By 2023-Oct-30, you need to update your project and remove deprecated calls
or your builds will no longer be supported.
See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
********************************************************************************
!!
parsed = self.parsers.get(option_name, lambda x: x)(value)
running egg_info
writing lib3/PyYAML.egg-info/PKG-INFO
writing dependency_links to lib3/PyYAML.egg-info/dependency_links.txt
writing top-level names to lib3/PyYAML.egg-info/top_level.txt
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-rlbi9ugx/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-rlbi9ugx/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
self.run_setup()
File "/tmp/pip-build-env-rlbi9ugx/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 338, in run_setup
exec(code, locals())
File "<string>", line 271, in <module>
File "/tmp/pip-build-env-rlbi9ugx/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 107, in setup
return distutils.core.setup(**attrs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-rlbi9ugx/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-rlbi9ugx/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/tmp/pip-build-env-rlbi9ugx/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/tmp/pip-build-env-rlbi9ugx/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 1234, in run_command
super().run_command(command)
File "/tmp/pip-build-env-rlbi9ugx/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/tmp/pip-build-env-rlbi9ugx/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 314, in run
self.find_sources()
File "/tmp/pip-build-env-rlbi9ugx/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 322, in find_sources
mm.run()
File "/tmp/pip-build-env-rlbi9ugx/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 551, in run
self.add_defaults()
File "/tmp/pip-build-env-rlbi9ugx/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 589, in add_defaults
sdist.add_defaults(self)
File "/tmp/pip-build-env-rlbi9ugx/overlay/lib/python3.11/site-packages/setuptools/command/sdist.py", line 104, in add_defaults
super().add_defaults()
File "/tmp/pip-build-env-rlbi9ugx/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
self._add_defaults_ext()
File "/tmp/pip-build-env-rlbi9ugx/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
self.filelist.extend(build_ext.get_source_files())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<string>", line 201, in get_source_files
File "/tmp/pip-build-env-rlbi9ugx/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
raise AttributeError(attr)
AttributeError: cython_sources
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
Γ Getting requirements to build wheel did not run successfully.
β exit code: 1
β°β> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
``
For the poetry users, when you run a command like the following in your Dockerfile, just make sure the pyyaml version matches whatever is in your poetry.lock file, otherwise poetry will try to install a new version and you get the same error.
poetry run pip install "Cython<3.0" "pyyaml==5.4.1" --no-build-isolation
For folks relying on aws-cli, pinning
pyyaml<=5.3.1helped us.NOTE - please read this note from aws-cli maintainers before you use
pyyaml<=5.3.1:
This did the trick for my builds. Thank you. I will keep an eye on this space for updates
I downgraded pyyaml version:
[tool.poetry.dependencies]
...
Cython = "<3.0.0"
pyyaml = "5.3.1"
It's working! For the time being. I think.
#607 fix my build for CPython 3.0.0, see alvistack@647faf1
A straightforward fix for Cython build problems is to use old_build_ext, which was present (and aliased to build_ext by default) in Cython 0.29.36. See ahesford/void-packages@dc86c6c for a working example, although it might be preferable to try this import first and fall back to the standard build_ext import should still older versions of Cython require it.
We were previously pinned to version 5.4 of PyYAML. This is working for us:
[tool.poetry.dependencies]
...
Cpython = "<3.0.0"
pyyaml = "5.4.1"
...
#702 was merged, 6.0.1 released, and this can be closed #702 (comment)
Is it possible to release a 5.4.2 version with Cpython restricted to "<3.0.0" ? This should solve some of the immediate problems related to 3rd party libraries pinning down PyYaml to versions lower than 6.0.
Is it possible to release a
5.4.2version with Cpython restricted to "<3.0.0" ? This should solve some of the immediate problems related to 3rd party libraries pinning down PyYaml to versions lower than 6.0.
This would help out a ton! Half of my CI builds are still wrecked by third-party dependencies that keep trying to install 5.4.1
This works if you compile from build file from GoHlke
Stumbled upon this bug while trying to install Pycaret:
Collecting PyYAML~=5.1 #0 30.24 Downloading PyYAML-5.4.1.tar.gz (175 kB) #0 30.24 ββββββββββββββββββββββββββββββββββββββ 175.1/175.1 kB 34.3 MB/s eta 0:00:00 #0 30.32 Installing build dependencies: started #0 33.08 Installing build dependencies: finished with status 'done' #0 33.08 Getting requirements to build wheel: started #0 33.19 Getting requirements to build wheel: finished with status 'error' #0 33.20 error: subprocess-exited-with-error #0 33.20 #0 33.20 Γ Getting requirements to build wheel did not run successfully. #0 33.20 β exit code: 1 #0 33.20 β°β> [50 lines of output] #0 33.20 /tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/config/setupcfg.py:516: SetuptoolsDeprecationWarning: T he license_file parameter is deprecated, use license_files instead. #0 33.20 warnings.warn(msg, warning_class) #0 33.20 running egg_info #0 33.20 writing lib3/PyYAML.egg-info/PKG-INFO #0 33.20 writing dependency_links to lib3/PyYAML.egg-info/dependency_links.txt #0 33.20 writing top-level names to lib3/PyYAML.egg-info/top_level.txt #0 33.20 Traceback (most recent call last): #0 33.20 File "/opt/conda/envs/rapids/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in < module> #0 33.20 main() #0 33.20 File "/opt/conda/envs/rapids/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in m ain #0 33.20 json_out['return_val'] = hook(**hook_input['kwargs']) #0 33.20 File "/opt/conda/envs/rapids/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in g et_requires_for_build_wheel #0 33.20 return hook(config_settings) #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in get_requires_for_b uild_wheel #0 33.20 return self._get_build_requires(config_settings, requirements=['wheel']) #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 320, in _get_build_require s #0 33.20 self.run_setup() #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 335, in run_setup #0 33.20 exec(code, locals()) #0 33.20 File "<string>", line 271, in <module> #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 108, in setup #0 33.20 return distutils.core.setup(**attrs) #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup #0 33.20 return run_commands(dist) #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands #0 33.20 dist.run_commands() #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands #0 33.20 self.run_command(cmd) #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1221, in run_command #0 33.20 super().run_command(command) #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command #0 33.20 cmd_obj.run() #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 318, in run #0 33.20 self.find_sources() #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 326, in find_sources #0 33.20 mm.run() #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 570, in run #0 33.20 self.add_defaults() #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 608, in add_defaults #0 33.20 sdist.add_defaults(self) #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/command/sdist.py", line 106, in add_defaults #0 33.20 super().add_defaults() #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults #0 33.20 self._add_defaults_ext() #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext #0 33.20 self.filelist.extend(build_ext.get_source_files()) #0 33.20 File "<string>", line 201, in get_source_files #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__ #0 33.20 raise AttributeError(attr) #0 33.20 AttributeError: cython_sources #0 33.20 [end of output]
Hi Any solution for this ? Thanks in advance
Stumbled upon this bug while trying to install Pycaret:
Collecting PyYAML~=5.1 #0 30.24 Downloading PyYAML-5.4.1.tar.gz (175 kB) #0 30.24 ββββββββββββββββββββββββββββββββββββββ 175.1/175.1 kB 34.3 MB/s eta 0:00:00 #0 30.32 Installing build dependencies: started #0 33.08 Installing build dependencies: finished with status 'done' #0 33.08 Getting requirements to build wheel: started #0 33.19 Getting requirements to build wheel: finished with status 'error' #0 33.20 error: subprocess-exited-with-error #0 33.20 #0 33.20 Γ Getting requirements to build wheel did not run successfully. #0 33.20 β exit code: 1 #0 33.20 β°β> [50 lines of output] #0 33.20 /tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/config/setupcfg.py:516: SetuptoolsDeprecationWarning: T he license_file parameter is deprecated, use license_files instead. #0 33.20 warnings.warn(msg, warning_class) #0 33.20 running egg_info #0 33.20 writing lib3/PyYAML.egg-info/PKG-INFO #0 33.20 writing dependency_links to lib3/PyYAML.egg-info/dependency_links.txt #0 33.20 writing top-level names to lib3/PyYAML.egg-info/top_level.txt #0 33.20 Traceback (most recent call last): #0 33.20 File "/opt/conda/envs/rapids/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in < module> #0 33.20 main() #0 33.20 File "/opt/conda/envs/rapids/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in m ain #0 33.20 json_out['return_val'] = hook(**hook_input['kwargs']) #0 33.20 File "/opt/conda/envs/rapids/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in g et_requires_for_build_wheel #0 33.20 return hook(config_settings) #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in get_requires_for_b uild_wheel #0 33.20 return self._get_build_requires(config_settings, requirements=['wheel']) #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 320, in _get_build_require s #0 33.20 self.run_setup() #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 335, in run_setup #0 33.20 exec(code, locals()) #0 33.20 File "<string>", line 271, in <module> #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 108, in setup #0 33.20 return distutils.core.setup(**attrs) #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup #0 33.20 return run_commands(dist) #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands #0 33.20 dist.run_commands() #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands #0 33.20 self.run_command(cmd) #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1221, in run_command #0 33.20 super().run_command(command) #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command #0 33.20 cmd_obj.run() #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 318, in run #0 33.20 self.find_sources() #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 326, in find_sources #0 33.20 mm.run() #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 570, in run #0 33.20 self.add_defaults() #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 608, in add_defaults #0 33.20 sdist.add_defaults(self) #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/command/sdist.py", line 106, in add_defaults #0 33.20 super().add_defaults() #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults #0 33.20 self._add_defaults_ext() #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext #0 33.20 self.filelist.extend(build_ext.get_source_files()) #0 33.20 File "<string>", line 201, in get_source_files #0 33.20 File "/tmp/pip-build-env-_n6pb0oj/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__ #0 33.20 raise AttributeError(attr) #0 33.20 AttributeError: cython_sources #0 33.20 [end of output]Hi Any solution for this ? Thanks in advance
Here is a workaround we are following which uses a constraints.txt to stop cython from going to 3.0.0 and allows our project to continue using PyYAML 5.4.1
migtools/mig-agnosticd#207 (comment)
$ cat constraints.txt
cython < 3.0
PIP_CONSTRAINT=constraints.txt pip3 install -r requirements.txt --no-cache
Thanks @jwmatthews I will check this , For now I have
pip install "cython<3.0.0" wheel && pip install pyyaml==5.4.1 --no-build-isolation, It seems my build happens fine. I will check if my tests pass
At the risk of repeating what has been said a dozen times already, I found this while trying to install docker-compose with pipx. This was my workaround:
$ echo "cython<3" > /tmp/constraint.txt
$ PIP_CONSTRAINT=/tmp/constraint.txt pipx install docker-composeThanks @jwmatthews, this works for me
$ cat constraints.txt
cython < 3.0
PIP_CONSTRAINT=constraints.txt pip3 install -r requirements.txt --no-cache
@r-asiebert
The specific issue here is if you're trying to install PyYAML on a system that doesn't have a wheel available. Cython is only involved when building from source. Most of the report are on Alpine because there aren't musl wheels available. If 6.x is working, my guess is you're using Python 3.10 or 3.11 which didn't have wheels available in the 5.4.1 release.
This will continue to be an issue until the build dependencies are properly constrained.I'm using python 11 on alpine on a Fresh environment using Poetry... But, as mentioned by kylebarron, this affects me anyways...
Totally solved on 6.0.1 for me
Totally solved on 6.0.1 for me
The same for me π Encountered this issue on 6.0, bumping version to 6.0.1 solved it.
Solution
pip install "Cython<3.0" pyyaml --no-build-isolation
pip install -r requirements.txt
For Solving this on Docker:
Include these 2 lines in your Dockerfile
RUN pip install "Cython<3.0" pyyaml --no-build-isolation
RUN pip install -r requirements.txt
I hope it solves your problem :)
This issue affects tensorflow/models object detection api installation and gives the same set of errors.
Downgrading pyyaml to 5.3.1 ( from 6.0.1 ) manually resolves it.
Solution
That is just a workaround and doesn't solve the underlying problem which is especially important if you can't downgrade cython because another project actually requires 3.0
I encountered this error with Pipenv... any help to workaround it is appreciated. Thanks
I do not know how to convert the workaround pip install "cython<3.0.0" wheel && pip install pyyaml==5.4.1 --no-build-isolation in Pipefile.
we want to use https://github.com/yaml/pyyaml for jupyterlab/notebook, sagemath/sage#36129, but we already moved to Cython 3, and we want all packages to build from source.
Here is an absolutely minimal patch that should allow both Cython 3 and 0.29:
https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-python/pyyaml/files/pyyaml-6.0.1-cython3.patch
Hello.
pyyaml 3.13, 5.3.1, 5.4.1, 6.0.1
py 2.7, 3.5, 3.8
python setup build
result:
python38 setup.py build
/usr/lib64/python3.8/distutils/dist.py:274: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
running build
running build_py
warning: build_py: byte-compiling is disabled, skipping.
running build_ext
Traceback (most recent call last):
File "setup.py", line 291, in <module>
setup(
File "/usr/lib64/python3.8/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib64/python3.8/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/lib64/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/lib64/python3.8/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/usr/lib64/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib64/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "setup.py", line 185, in run
_build_ext.run(self)
File "/usr/lib64/python3.8/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "setup.py", line 227, in build_extensions
ext.sources = self.cython_sources(ext.sources, ext)
File "/usr/lib64/python3.8/distutils/cmd.py", line 103, in __getattr__
raise AttributeError(attr)
AttributeError: cython_sources
all
This normal?
rpm -qa | grep cython
python-cython-3.0.4-1-rosa2016.1.x86_64
python3-cython-3.0.4-1-rosa2016.1.x86_64
python38-cython-3.0.4-1-rosa2016.1.x86_64
IMHO, cython should be listed as permanent dependency of pyyaml if installing it might fail when it is missing, otherwise is a source of pain for the users.
Also this translates to other problems down the line because if that dependency is not listed, people would have to create an additional step to pre-install 'hidden' dependencies before installing normal ones, as installation order might make it fail otherwise.
I observed this bug not only with latest pyyaml 3.x but also on macos and ubuntu.
Two workarounds exist:
- Preinstall
cython<3, then install pyyaml without build isolation, then install the rest of your dependencies #601 (comment)
$ pip install "cython<3.0.0" wheel
$ pip install "pyyaml==5.4.1" --no-build-isolation
$ pip install -r requirements.txt
- Use a constraints file to force pip to use
cython<3at build time #601 (comment)
$ echo "cython<3" > /tmp/constraint.txt
$ PIP_CONSTRAINT=/tmp/constraint.txt pip install -r requirements.txt
Two workarounds exist:
- Preinstall
cython<3, then install pyyaml without build isolation, then install the rest of your dependencies "AttributeError: cython_sources" with Cython 3.0.0a10 #601 (comment)$ pip install "cython<3.0.0" wheel $ pip install "pyyaml==5.4.1" --no-build-isolation $ pip install -r requirements.txt
- Use a constraints file to force pip to use
cython<3at build time "AttributeError: cython_sources" with Cython 3.0.0a10 #601 (comment)$ echo "cython<3" > /tmp/constraint.txt $ PIP_CONSTRAINT=/tmp/constraint.txt pip install -r requirements.txt
Thank you. The first approach worked for me
Two workarounds exist:
* Preinstall `cython<3`, then install pyyaml without build isolation, then install the rest of your dependencies ["AttributeError: cython_sources" with Cython 3.0.0a10 #601 (comment)](https://github.com/yaml/pyyaml/issues/601#issuecomment-1660456820)$ pip install "cython<3.0.0" wheel $ pip install "pyyaml==5.4.1" --no-build-isolation $ pip install -r requirements.txt* Use a constraints file to force pip to use `cython<3` at build time ["AttributeError: cython_sources" with Cython 3.0.0a10 #601 (comment)](https://github.com/yaml/pyyaml/issues/601#issuecomment-1661960662)$ echo "cython<3" > /tmp/constraint.txt $ PIP_CONSTRAINT=/tmp/constraint.txt pip install -r requirements.txt
Thanks!
Any solution when working with pdm package manager?
Two workarounds exist:
- Preinstall
cython<3, then install pyyaml without build isolation, then install the rest of your dependencies "AttributeError: cython_sources" with Cython 3.0.0a10 #601 (comment)$ pip install "cython<3.0.0" wheel $ pip install "pyyaml==5.4.1" --no-build-isolation $ pip install -r requirements.txt
- Use a constraints file to force pip to use
cython<3at build time "AttributeError: cython_sources" with Cython 3.0.0a10 #601 (comment)$ echo "cython<3" > /tmp/constraint.txt $ PIP_CONSTRAINT=/tmp/constraint.txt pip install -r requirements.txt
same, this worked for me as well. thanks!
Version 6.0.1 now seems to require Cython < 3.0.0, so the workaround isn't required anymore I believe.
Can we change the setup.py to ignore cython when not need it? I installed cython for my own code, I don't want to use cython to compile pyyaml.
Suggest changes in https://github.com/yaml/pyyaml/blob/main/setup.py#L79C1-L89C14
with_cython = False
if 'sdist' in sys.argv or os.environ.get('PYYAML_FORCE_CYTHON') == '1':
# we need cython here
with_cython = True
if with_cython:
from Cython.Distutils.extension import Extension as _Extension
from Cython.Distutils import build_ext as _build_ext@JannKleen yes, however, as it appears, some users are installing pyyaml through first installing cython and then using --no-build-isolation. This will bypass the build dependency requirement written into pyproject.toml and reproduce the original problem.
For anyone who's struggling with the issue @nanonyme mentioned (installing current Cython first and using --no-build-isolation to install PyYAML), I've found that patching line 85 of the PyYAML setup.py script from
from Cython.Distutils import build_ext as _build_extto
from Cython.Distutils.old_build_ext import old_build_ext as _build_extbefore installing works for me with Cython 3.0.7 and PyYAML 6.0.1. It seems that the old build_ext was replaced in Cython but still lives in old_build_ext.py. That's probably a temporary solution, though, until old_build_ext will be removed.
Edit: I see that option is discussed (and rejected for long-term use) in cython/cython#4568.
mjziebarth's hint was great, but I am seeing this
DEPRECATION: Loading egg at /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/PyYAML-6.0.1-py3.12-macosx-10.9-universal2.egg is deprecated.
pip 24.3 will enforce this behaviour change.
A possible replacement is to use pip for package installation..
Discussion can be found at https://github.com/pypa/pip/issues/12330
Well, yes, clearly there is no other long-term solution than making pyyaml support Cython 3.
and stop using outdated distutils, as well. Use instead setuptools, or meson.
Try thsοΌ
pip install PyYaml==3.10
Sorry for commenting on someone else but DO NOT USE @yinxs2003 solution without considerations. It will install more than a 10-year-old version. In the meantime, PyYAML has at least one critical vulnerability fixed.
@yinxs2003 If You could at least inform people of the danger of using "older" code it would be great.
Managed to get PyYAML compile with python 3.12 by adding Cython 0.29.37 at the beginning of PYTHONPATH.
It's not necessarily to document workarounds. You can take pip or pypa/build and keep build isolation on. (which is the default) This will install compatible dependencies into virtualenv/venv to produce a wheel. You can then install wheel outside virtualenv/venv. This issue is about fixing compatibility with new Cython.