CMakeNotFoundError with pyodide: cmake -E capabilities failed
pthom opened this issue · 8 comments
Hello,
I am brand new to pyodide, but I'm trying to compile a project (imgui_bundle) that uses scikit-build-core.
I'm trying to build it from inside the pyodide build tree, and inside the Docker container, by running this command:
export CMAKE_EXECUTABLE=$(which cmake) # failed attempt to help cmake.py find cmake
PYODIDE_PACKAGES="imgui_bundle" makeAnd I get this error:
File "/tmp/build-env-azbe70tj/lib/python3.12/site-packages/scikit_build_core/build/__init__.py", line 31, in build_wheel
return _build_wheel_impl(
^^^^^^^^^^^^^^^^^^
File "/tmp/build-env-azbe70tj/lib/python3.12/site-packages/scikit_build_core/build/wheel.py", line 151, in _build_wheel_impl
cmake = CMake.default_search(version=settings.cmake.version, env=os.environ)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/build-env-azbe70tj/lib/python3.12/site-packages/scikit_build_core/cmake.py", line 63, in default_search
raise CMakeNotFoundError(msg)
scikit_build_core.errors.CMakeNotFoundError: Could not find CMake with version >=3.26.1
ERROR Backend subprocess exited when trying to invoke build_wheel
[2024-04-19 18:51:33] Failed building package imgui_bundle in 2.4 seconds.
ERROR: cancelling buildall due to error building imgui_bundle
Here is an extract from my pyproject.toml:
[build-system]
requires = ["scikit-build-core", "pybind11"]
build-backend = "scikit_build_core.build"
[project]
name = "imgui-bundle"
version = "1.5.0"
requires-python = ">=3.10"
dependencies = [ "numpy", "munch", "glfw", "PyOpenGL", "pillow"]
...
[tool.scikit-build]
wheel.expand-macos-universal-tags = true
wheel.packages = ["bindings/imgui_bundle"]
cmake.version = ">=3.26.1"
cmake.verbose = true
logging.level = "INFO"And the meta.yaml file I am using:
package:
name: imgui_bundle
version: 1.3.0
top-level:
- imgui_bundle
source:
path: /workspaces/pyodide/_bundle/imgui_bundle
about:
home: https://github.com/pthom/imgui_bundle
PyPI: https://pypi.org/project/imgui_bundle
summary:
"Dear ImGui Bundle: easily create ImGui applications in Python and C++.
Batteries included!"
license: ""
extra:
recipe-maintainers:
- pthom
requirements:
# host:
# #Dependencies that are needed to build the package
# - scikit-build-core>=0.3.3
run:
# Dependencies that are needed to run the package
- numpy
- munch
- PillowI did find where the exception is raised:
scikit-build-core/src/scikit_build_core/cmake.py
Lines 53 to 63 in f844088
It seems that it might perhaps be possible to pass the cmake location via the env variable CMAKE_EXECUTABLE, but this does not work.
I'm afraid I might be missing something obvious. However, I can not find a way to debug the internals of scikit_build_core, Since it is installed in a temporary isolated environment. Help would be appreciated.
Many thanks!
I was able to debug inside scikit-build-core, by using backend-path inside pyproject.toml (cf cf https://peps.python.org/pep-0517/#in-tree-build-backends)
[build-system]
requires = ["pybind11", "packaging", "pathspec"]
build-backend = "scikit_build_core.build"
backend-path = ["_backends/scikit-build-core/src/"]As a matter of fact get_cmake_program will call cmake -E capabilities which fails.
I Instrumented its code as below:
def get_cmake_program(cmake_path: Path) -> Program:
"""
Get the Program (with version) for CMake given a path. The version will be
None if it cannot be determined.
"""
# get version and capabilities with subprocess (my instrumentation)
import subprocess
version_result = subprocess.run([cmake_path, "--version"], capture_output=True)
capabilities_result = subprocess.run([cmake_path, "-E", "capabilities"], capture_output=True)
raise Exception(f"""
get_cmake_program
{cmake_path=}
{version_result.stdout=}
{version_result.returncode=})
{capabilities_result.stdout=}
{capabilities_result.returncode=}
{capabilities_result.stderr=}
)
""")
# original code below
try:
result = Run().capture(cmake_path, "-E", "capabilities")
except (subprocess.CalledProcessError, PermissionError):
return Program(cmake_path, None)
try:
version = Version(json.loads(result.stdout)["version"]["string"])
except (json.decoder.JSONDecodeError, KeyError, InvalidVersion):
logger.warning("Could not determine CMake version, got {!r}", result.stdout)
return Program(cmake_path, None)
logger.info("CMake version: {}", version)
return Program(cmake_path, version)And I get the following exception as an output:
Exception:
get_cmake_program
cmake_path=PosixPath('/tmp/tmpxg_inkrx/cmake')
version_result.stdout=b'cmake version 3.28.3\n\nCMake suite maintained and supported by Kitware
(kitware.com/cmake).\n'
version_result.returncode=0)
capabilities_result.stdout=b''
capabilities_result.returncode=1
capabilities_result.stderr=b"configure: cmake -DCMAKE_C_COMPILER=/tmp/tmpxg_inkrx/cc
-DCMAKE_CXX_COMPILER=/tmp/tmpxg_inkrx/c++ -DCMAKE_AR=/tmp/tmpxg_inkrx/ar -DCMAKE_C_COMPILER_AR=/tmp/tmpxg_inkrx/ar
-DCMAKE_CXX_COMPILER_AR=/tmp/tmpxg_inkrx/ar --fresh -E capabilities
-DCMAKE_CROSSCOMPILING_EMULATOR=/workspaces/pyodide/emsdk/emsdk/node/16.20.0_64bit/bin/node\nCMake Error: Unknown argument -E\nCMake
Error: Run 'cmake --help' for all supported options.\nemcmake: error: 'cmake -DCMAKE_C_COMPILER=/tmp/tmpxg_inkrx/cc
-DCMAKE_CXX_COMPILER=/tmp/tmpxg_inkrx/c++ -DCMAKE_AR=/tmp/tmpxg_inkrx/ar -DCMAKE_C_COMPILER_AR=/tmp/tmpxg_inkrx/ar
-DCMAKE_CXX_COMPILER_AR=/tmp/tmpxg_inkrx/ar --fresh -E capabilities
-DCMAKE_CROSSCOMPILING_EMULATOR=/workspaces/pyodide/emsdk/emsdk/node/16.20.0_64bit/bin/node' failed (returned 1)\n"
)
capabilities_result.stderr is a bit strange since it mixes errrors from elsewhere. However it contains this interesting piece:
CMake Error: Unknown argument -E
CMake Error: Run 'cmake --help' for all supported options.
So as a conclusion, I can see that the command cmake -E capabilities might fail (perhaps because some compilation options of CMake do not support this kind of output), even with cmake 3.28.3
Probably #715?
oh yes, after a deep dive in the internal of the code, I now see that it it the same.
I'll be fixing shortly. Really pyodide's fault, but I can fix here faster.
(0.8.x works until then with Pyodide)
Fix released as 0.9.1!
Thanks a lot !!!