pre-commit not compatible with hooks that require compiled dependencies (e.g. ansible-lint)
m-roberts opened this issue · 2 comments
m-roberts commented
When trying to use this config:
repos:
- repo: https://github.com/ansible-community/ansible-lint.git
rev: v5.0.7
hooks:
- id: ansible-lint
files: \.(yaml|yml)$
I get this:
[INFO] Initializing environment for https://github.com/ansible-community/ansible-lint.git.
[INFO] Initializing environment for https://github.com/ansible-community/ansible-lint.git:.[community,yamllint].
[INFO] Installing environment for https://github.com/ansible-community/ansible-lint.git.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
An unexpected error has occurred: CalledProcessError: command: ('/root/.cache/pre-commit/reponmshav7x/py_env-python3.9/bin/python', '-mpip', 'install', '.', '.[community,yamllint]')
return code: 1
expected return code: 0
stdout:
Processing /root/.cache/pre-commit/reponmshav7x
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing wheel metadata: started
Preparing wheel metadata: finished with status 'done'
Processing /root/.cache/pre-commit/reponmshav7x
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing wheel metadata: started
Preparing wheel metadata: finished with status 'done'
Collecting packaging
Downloading packaging-20.9-py2.py3-none-any.whl (40 kB)
Collecting ruamel.yaml<1,>=0.15.37; python_version >= "3.7"
Using cached ruamel.yaml-0.17.4-py3-none-any.whl (101 kB)
Collecting pyyaml
Downloading PyYAML-5.4.1.tar.gz (175 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing wheel metadata: started
Preparing wheel metadata: finished with status 'done'
Collecting tenacity
Downloading tenacity-7.0.0-py2.py3-none-any.whl (23 kB)
Collecting enrich>=1.2.6
Downloading enrich-1.2.6-py3-none-any.whl (8.6 kB)
Collecting rich>=9.5.1
Downloading rich-10.1.0-py3-none-any.whl (201 kB)
Collecting wcmatch>=7.0
Downloading wcmatch-8.1.2-py3-none-any.whl (40 kB)
Collecting ansible>=2.10; extra == "community"
Downloading ansible-3.2.0.tar.gz (31.3 MB)
Collecting yamllint>=1.25.0; extra == "yamllint"
Downloading yamllint-1.26.1.tar.gz (126 kB)
Collecting pyparsing>=2.0.2
Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Processing /root/.cache/pip/wheels/4d/ec/94/94b660639d0a3d73c59770481208fb98c1d8dd38c87d246698/ruamel.yaml.clib-0.2.2-py3-none-any.whl
Collecting six>=1.9.0
Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting pygments<3.0.0,>=2.6.0
Downloading Pygments-2.8.1-py3-none-any.whl (983 kB)
Collecting colorama<0.5.0,>=0.4.0
Downloading colorama-0.4.4-py2.py3-none-any.whl (16 kB)
Collecting commonmark<0.10.0,>=0.9.0
Downloading commonmark-0.9.1-py2.py3-none-any.whl (51 kB)
Collecting typing-extensions<4.0.0,>=3.7.4
Downloading typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Collecting bracex>=2.1.1
Downloading bracex-2.1.1-py3-none-any.whl (10 kB)
Collecting ansible-base<2.11,>=2.10.7
Downloading ansible-base-2.10.7.tar.gz (5.7 MB)
Collecting pathspec>=0.5.3
Downloading pathspec-0.8.1-py2.py3-none-any.whl (28 kB)
Collecting jinja2
Downloading Jinja2-2.11.3-py2.py3-none-any.whl (125 kB)
Collecting cryptography
Downloading cryptography-3.4.7.tar.gz (546 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'error'
stderr:
ERROR: Command errored out with exit status 1:
command: /root/.cache/pre-commit/reponmshav7x/py_env-python3.9/bin/python /root/.cache/pre-commit/reponmshav7x/py_env-python3.9/lib/python3.9/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-u7scfaiu/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.6.0' wheel 'cffi>=1.12; platform_python_implementation != '"'"'PyPy'"'"'' 'setuptools-rust>=0.11.4'
cwd: None
Complete output (124 lines):
Collecting setuptools>=40.6.0
Using cached setuptools-54.2.0-py3-none-any.whl (785 kB)
Collecting wheel
Using cached wheel-0.36.2-py2.py3-none-any.whl (35 kB)
Collecting cffi>=1.12
Downloading cffi-1.14.5.tar.gz (475 kB)
Collecting setuptools-rust>=0.11.4
Downloading setuptools_rust-0.12.1-py3-none-any.whl (22 kB)
Collecting pycparser
Downloading pycparser-2.20-py2.py3-none-any.whl (112 kB)
Collecting toml>=0.9.0
Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting semantic-version>=2.6.0
Downloading semantic_version-2.8.5-py2.py3-none-any.whl (15 kB)
Building wheels for collected packages: cffi
Building wheel for cffi (setup.py): started
Building wheel for cffi (setup.py): finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /root/.cache/pre-commit/reponmshav7x/py_env-python3.9/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-_1btoeh2/cffi/setup.py'"'"'; __file__='"'"'/tmp/pip-install-_1btoeh2/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-0nu13nh7
cwd: /tmp/pip-install-_1btoeh2/cffi/
Complete output (45 lines):
No working compiler found, or bogus compiler options passed to
the compiler from Python's standard "distutils" module. See
the error messages above. Likely, the problem is not related
to CFFI but generic to the setup.py of any Python package that
tries to compile C code. (Hints: on OS/X 10.8, for errors about
-mno-fused-madd see http://stackoverflow.com/questions/22313407/
Otherwise, see https://wiki.python.org/moin/CompLangPython or
the IRC channel #python on irc.freenode.net.)
Trying to continue anyway. If you are trying to install CFFI from
a build done in a different context, you can ignore this warning.
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.9
creating build/lib.linux-x86_64-3.9/cffi
copying cffi/verifier.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/lock.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/cparser.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/vengine_gen.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/api.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/__init__.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/model.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/cffi_opcode.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/setuptools_ext.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/pkgconfig.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/backend_ctypes.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/error.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/ffiplatform.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/vengine_cpy.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/commontypes.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/recompiler.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/_cffi_include.h -> build/lib.linux-x86_64-3.9/cffi
copying cffi/parse_c_type.h -> build/lib.linux-x86_64-3.9/cffi
copying cffi/_embedding.h -> build/lib.linux-x86_64-3.9/cffi
copying cffi/_cffi_errors.h -> build/lib.linux-x86_64-3.9/cffi
running build_ext
building '_cffi_backend' extension
creating build/temp.linux-x86_64-3.9
creating build/temp.linux-x86_64-3.9/c
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -DTHREAD_STACK_SIZE=0x100000 -fPIC -I/usr/include/ffi -I/usr/include/libffi -I/root/.cache/pre-commit/reponmshav7x/py_env-python3.9/include -I/usr/local/include/python3.9 -c c/_cffi_backend.c -o build/temp.linux-x86_64-3.9/c/_cffi_backend.o
error: command 'gcc' failed: No such file or directory
----------------------------------------
ERROR: Failed building wheel for cffi
Running setup.py clean for cffi
Failed to build cffi
Installing collected packages: setuptools, wheel, pycparser, cffi, toml, semantic-version, setuptools-rust
Running setup.py install for cffi: started
Running setup.py install for cffi: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /root/.cache/pre-commit/reponmshav7x/py_env-python3.9/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-_1btoeh2/cffi/setup.py'"'"'; __file__='"'"'/tmp/pip-install-_1btoeh2/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-yx_q6xtg/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-u7scfaiu/overlay --compile --install-headers /tmp/pip-build-env-u7scfaiu/overlay/include/site/python3.9/cffi
cwd: /tmp/pip-install-_1btoeh2/cffi/
Complete output (45 lines):
No working compiler found, or bogus compiler options passed to
the compiler from Python's standard "distutils" module. See
the error messages above. Likely, the problem is not related
to CFFI but generic to the setup.py of any Python package that
tries to compile C code. (Hints: on OS/X 10.8, for errors about
-mno-fused-madd see http://stackoverflow.com/questions/22313407/
Otherwise, see https://wiki.python.org/moin/CompLangPython or
the IRC channel #python on irc.freenode.net.)
Trying to continue anyway. If you are trying to install CFFI from
a build done in a different context, you can ignore this warning.
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.9
creating build/lib.linux-x86_64-3.9/cffi
copying cffi/verifier.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/lock.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/cparser.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/vengine_gen.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/api.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/__init__.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/model.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/cffi_opcode.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/setuptools_ext.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/pkgconfig.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/backend_ctypes.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/error.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/ffiplatform.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/vengine_cpy.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/commontypes.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/recompiler.py -> build/lib.linux-x86_64-3.9/cffi
copying cffi/_cffi_include.h -> build/lib.linux-x86_64-3.9/cffi
copying cffi/parse_c_type.h -> build/lib.linux-x86_64-3.9/cffi
copying cffi/_embedding.h -> build/lib.linux-x86_64-3.9/cffi
copying cffi/_cffi_errors.h -> build/lib.linux-x86_64-3.9/cffi
running build_ext
building '_cffi_backend' extension
creating build/temp.linux-x86_64-3.9
creating build/temp.linux-x86_64-3.9/c
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -DTHREAD_STACK_SIZE=0x100000 -fPIC -I/usr/include/ffi -I/usr/include/libffi -I/root/.cache/pre-commit/reponmshav7x/py_env-python3.9/include -I/usr/local/include/python3.9 -c c/_cffi_backend.c -o build/temp.linux-x86_64-3.9/c/_cffi_backend.o
error: command 'gcc' failed: No such file or directory
----------------------------------------
ERROR: Command errored out with exit status 1: /root/.cache/pre-commit/reponmshav7x/py_env-python3.9/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-_1btoeh2/cffi/setup.py'"'"'; __file__='"'"'/tmp/pip-install-_1btoeh2/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-yx_q6xtg/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-u7scfaiu/overlay --compile --install-headers /tmp/pip-build-env-u7scfaiu/overlay/include/site/python3.9/cffi Check the logs for full command output.
----------------------------------------
ERROR: Command errored out with exit status 1: /root/.cache/pre-commit/reponmshav7x/py_env-python3.9/bin/python /root/.cache/pre-commit/reponmshav7x/py_env-python3.9/lib/python3.9/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-u7scfaiu/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.6.0' wheel 'cffi>=1.12; platform_python_implementation != '"'"'PyPy'"'"'' 'setuptools-rust>=0.11.4' Check the logs for full command output.
Check the log at /root/.cache/pre-commit/pre-commit.log
Smoking gun: error: command 'gcc' failed: No such file or directory
This Dockerfile does not have gcc installed, and so therefore cannot be used with this action, or any that suffer the same issue.
tjamet commented
thanks for raising.
I will try to take a look sometime soon
tjamet commented
Sorry about the delay taking a look at the issue.
The problem here is that some python modules need to be installed to be able to be run.
Usually, pre-commit is able to do it on a single machine and this is not a big deal because it is done only once.
Running pre-commit in a container would mean that the container is re-built every time pre-commit is run. Hence probably too often.
We could evaluate whether adding the relevant volume bindings to the pre-commit image and gcc would help. But I tend to foresee complications moving forward.
My suggestion here would be to try to pre-bake an image with all the pre-commit tools inside it. Not sure how to do so though.
Hope this helps