Add `--jobs` option to `build`
YoSTEALTH opened this issue · 5 comments
Add --jobs option to build to use multiple threads. Currently compiling takes a long time in single thread. For example python3 setup.py build_ext -j16 --inplace already exists.
Build has no control over the backend other than passing through config settings to it. Check with your backend. Scikit-build-core supports pipx run build -Ccmake.define.CMAKE_BUILD_PARALLEL_LEVEL=8 for example (and defaults to parallel builds).
Currently I am trying to test build for https://github.com/YoSTEALTH/Liburing/blob/master/setup.py#L61 I do add how many thread to use. Running it as python3 -m build
Setuptools only supports compiling extensions in parallel, not individual files. You have to hack it in with something like https://github.com/pybind/pybind11/blob/7f2214bc680b7cc9f6ad138a077b3d6d172b51f7/pybind11/setup_helpers.py#L367-L500 - or you can use a modern build system like scikit-build-core or meson-python.
Looks like python3 -m build -C--global-option="build_ext -j16" solves the problem, mentioned in #328
It would be nice if build could expose or take advantage of such features. Maybe add build config in pyproject.toml or something.
This is up to the backend. Scikit-build-core does let you set this in pyproject.toml. Setuptools could add it.