p4p incompatible with NumPy 2.0
Opened this issue · 11 comments
DiamondJoseph commented
On attempting to load p4p with the latest release of NumPy raises errors due to being compiled against the previous version.
File "src/p4p/_p4p.pyx", line 1, in init p4p._p4p
ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
mdavidsaver commented
Oh goody... Any idea how to accommodate both numpy 1.x and 2.x series without (somehow) doubling the .whl files?
fyi. the error comes from cython.
callumforrester commented
Unsure... @coretl might have some ideas...
AlexanderWells-diamond commented
I've done some digging and it looks like Numpy is pulling some magic under the covers to make 2.0 builds compatible with 1.* builds. They have a guide here, which seems to simply say that you build against 2.* and it'll work for 1.*. There's a caveat about what Python versions it may work with though; Numpy 2.0 is Python3.9+, so care will have to be taken to continue supporting older Python versions.
tacaswell commented
For reference, look at how scipy or Matplotlib are doing this.
For unsupported versions of Python (<=3.8) you will need to fallback to oldest-supported-numpy to build the wheels with a version of numpy that will work with any numpy that has a wheels for that Python version.
finnoshea commented
p4p now seems to require numpy 2.0. Is this intended? It appears to make p4p impossible to install.
From my latest attempts to install p4p:
67.85 The conflict is caused by:
67.85 The user requested numpy<1.23
67.85 p4p 3.5.0 depends on numpy>=2.0.1
This shows up in setup.py:
install_requires = [
epicscorelibs.version.abi_requires(),
pvxslibs.version.abi_requires(),
# assume ABI forward compatibility as indicated by
# https://github.com/numpy/numpy/blob/master/numpy/core/setup_common.py#L28
'numpy >=%s'%numpy.version.short_version,
'nose2>=0.8.0',
'ply', # for asLib
]
I wonder if this is inheriting a numpy 2.0 dependence somehow?
Yep, the problem is here. The problem code appears to have been committed by @OCopping 10 hours ago.
mdavidsaver commented
@finnoshea Which version of python are you using? Which platform? Do you expect/want to use a pre-built wheel?
I think you are correct that the 'numpy >=%s'%numpy.version.short_version, dependency is now too strict post numpy 2.x.
The original logic was to build wheels against a fairly old numpy, which was in practice ABI compatible with newer versions at runtime.
For numpy 2.x, I'm not sure what the oldest backwards compatible version is wrt. ABI.
install_requires = [ ... ] # existing deps excluding numpy
if numpy.lib.NumpyVersion(numpy.__version__) >= '2.0.0b1':
install_requires += ['numpy >= 1.7', 'numpy < 3']
else:
install_requires += ['numpy >=%s'%numpy.version.short_version, 'numpy < 2']I pick 1.7 somewhat arbitrarily. I think that whichever version is picked will probably need to inject eg. :
#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSIONtacaswell commented
Please read the manual (also already linked above): https://numpy.org/devdocs/dev/depending_on_numpy.html#adding-a-dependency-on-numpy
mdavidsaver commented
Please read the manual ...
I did last month, and I have again now. I admit that I was confusing the existing NPY_NO_DEPRECATED_API with the new NPY_TARGET_VERSION macro. So the answer to my question of what the oldest possible backwards compatible version for 2.x builds is in fact 1.7.
This document does not address also supporting source builds against 1.x, which I think P4P needs to support. So I think the condition I wrote above will be necessary, with the addition of defining the per-processor macro NPY_TARGET_VERSION to NPY_1_7_API_VERSION.
finnoshea commented
Apologies for the verbose dump in this comment. I've been trying to install p4p in a Docker container.
Here is the Dockerfile I inherited:
ARG PYTHON_VERSION=3.10-slim-bullseye
FROM python:${PYTHON_VERSION}
SHELL ["/bin/bash", "-c"]
ARG DEBIAN_FRONTEND=noninteractive
ENV PYTHONFAULTHANDLER=1 \
PYTHONUNBUFFERED=1 \
PYTHONHASHSEED=random \
PIP_NO_CACHE_DIR=off \
PIP_DISABLE_PIP_VERSION_CHECK=on \
PIP_DEFAULT_TIMEOUT=100
# Install build dependencies
RUN apt-get update \
&& apt-get install -y --no-install-recommends build-essential curl git vim poppler-utils pkg-config libhdf5-serial-dev \
&& apt-get autoremove -y \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
# Install poetry as package manager
#ENV POETRY_HOME="/opt/poetry" \
# POETRY_VIRTUALENVS_IN_PROJECT=1 \
# POETRY_NO_INTERACTION=1
#ARG POETRY_VERSION=1.4.1
#RUN curl -sSL https://install.python-poetry.org | POETRY_HOME=${POETRY_HOME} POETRY_VERSION=${POETRY_VERSION} python3 -
# Setup venv
RUN python3 -m venv /opt/venv
RUN pip3 install --upgrade pip setuptools wheel
# Install packages
#ENV PATH="/opt/venv/bin:$POETRY_HOME/bin:$PATH"
WORKDIR /opt/SLAC-AD
RUN source /opt/venv/bin/activate
#COPY pyproject.toml poetry.lock ./
RUN source /opt/venv/bin/activate
#RUN poetry config virtualenvs.create false \
# && poetry install --no-interaction --no-ansi --no-root
COPY requirements.txt ./
RUN pip3 install -r requirements.txt
COPY p4p_reqs.txt ./
RUN pip3 install -r p4p_reqs.txt
#ADD p4p_install /opt/p4p
#RUN pip3 install epicscorelibs pvxslibs /opt/p4p
# Set env var needed to load EPICS data (as expected by the meme/p4p packages)
# Valid only on machines where mccas0 is a known alias
ENV EPICS_PVA_ADDR_LIST=mccas0
# Set env var needed to load BSA data
# Requires the BSA data directory to be mounted to this path
ENV BSA_DATA_DIR=/opt/data/bsa
# Set env var to project home
# Requires the project home directory to be mounted to this path
ENV PROJECT_HOME=/opt/project
WORKDIR /opt/SLAC-AD
I've had to excise the poetry stuff from there, because it does not play well with pytorch. The two files of interest to you are requirements.txt:
colorama #==0.4.6 ; python_version >= "3.10" and python_version < "3.11" and platform_system == "Windows"
#epicscorelibs==7.0.7.99.1.1a2 #==7.0.7.99.0.2 ; python_version >= "3.10" and python_version < "3.11"
h5py #==3.11.0 ; python_version >= "3.10" and python_version < "3.11"
joblib #==1.4.2 ; python_version >= "3.10" and python_version < "3.11"
kaleido #==0.2.1 ; python_version >= "3.10" and python_version < "3.11"
llvmlite #==0.39.1 ; python_version >= "3.10" and python_version < "3.11"
#meme @ git+https://github.com/slaclab/meme@af07afe431fa4503664aca8d9f83f786f85abbf0 ; python_version >= "3.10" and python_version < "3.11"
#nose2 #==0.15.1 ; python_version >= "3.10" and python_version < "3.11"
numba #==0.56.4 ; python_version >= "3.10" and python_version < "3.11"
numpy #==1.23.5 ; python_version >= "3.10" and python_version < "3.11"
#p4p #4.1.12 ; python_version >= "3.10" and python_version < "3.11"
packaging #==24.1 ; python_version >= "3.10" and python_version < "3.11"
pandas #==1.5.3 ; python_version >= "3.10" and python_version < "3.11"
plotly #==5.23.0 ; python_version >= "3.10" and python_version < "3.11"
ply #==3.11 ; python_version >= "3.10" and python_version < "3.11"
#pvxslibs #==1.3.1 ; python_version >= "3.10" and python_version < "3.11"
pynndescent #==0.5.13 ; python_version >= "3.10" and python_version < "3.11"
python-dateutil #==2.9.0.post0 ; python_version >= "3.10" and python_version < "3.11"
pytz #==2022.7.1 ; python_version >= "3.10" and python_version < "3.11"
scikit-learn #==1.5.1 ; python_version >= "3.10" and python_version < "3.11"
scipy #==1.14.0 ; python_version >= "3.10" and python_version < "3.11"
setuptools-dso #==2.10 ; python_version >= "3.10" and python_version < "3.11"
setuptools #==72.2.0 ; python_version >= "3.10" and python_version < "3.11"
six #==1.16.0 ; python_version >= "3.10" and python_version < "3.11"
tenacity #==9.0.0 ; python_version >= "3.10" and python_version < "3.11"
threadpoolctl #==3.5.0 ; python_version >= "3.10" and python_version < "3.11"
tqdm #==4.66.5 ; python_version >= "3.10" and python_version < "3.11"
umap-learn #==0.5.6 ; python_version >= "3.10" and python_version < "3.11"
xlrd #==2.0.1 ; python_version >= "3.10" and python_version < "3.11"
and p4p_reqs.txt:
epicscorelibs
meme @ git+https://github.com/slaclab/meme@af07afe431fa4503664aca8d9f83f786f85abbf0 ; python_version >= "3.10" and python_version < "3.11"
nose2 #==0.15.1 ; python_version >= "3.10" and python_version < "3.11"
p4p #4.1.12 ; python_version >= "3.10" and python_version < "3.11"
Below is the error I get when trying to pip3 install p4p_reqs.txt. I get the same error with 3.11-slim-bullseye as the base. If I limit numpy to numpy<1.23, it appears to download 2.0.1 anyway and then I get the same error.
0.207 Collecting meme@ git+https://github.com/slaclab/meme@af07afe431fa4503664aca8d9f83f786f85abbf0 (from -r p4p_reqs.txt (line 2))
0.208 Cloning https://github.com/slaclab/meme (to revision af07afe431fa4503664aca8d9f83f786f85abbf0) to /tmp/pip-install-12gmrgwt/meme_66b649dcaeb54ab2ab297066b8f9da80
0.208 Running command git clone --filter=blob:none --quiet https://github.com/slaclab/meme /tmp/pip-install-12gmrgwt/meme_66b649dcaeb54ab2ab297066b8f9da80
1.209 Running command git rev-parse -q --verify 'sha^af07afe431fa4503664aca8d9f83f786f85abbf0'
1.210 Running command git fetch -q https://github.com/slaclab/meme af07afe431fa4503664aca8d9f83f786f85abbf0
1.483 Running command git checkout -q af07afe431fa4503664aca8d9f83f786f85abbf0
1.831 Resolved https://github.com/slaclab/meme to commit af07afe431fa4503664aca8d9f83f786f85abbf0
1.835 Preparing metadata (setup.py): started
1.944 Preparing metadata (setup.py): finished with status 'done'
2.118 Collecting epicscorelibs (from -r p4p_reqs.txt (line 1))
2.227 Downloading epicscorelibs-7.0.7.99.0.2.tar.gz (1.6 MB)
2.289 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 116.5 MB/s eta 0:00:00
2.432 Installing build dependencies: started
3.266 Installing build dependencies: finished with status 'done'
3.267 Getting requirements to build wheel: started
3.405 Getting requirements to build wheel: finished with status 'done'
3.406 Preparing metadata (pyproject.toml): started
3.529 Preparing metadata (pyproject.toml): finished with status 'done'
3.542 Collecting nose2 (from -r p4p_reqs.txt (line 3))
3.548 Downloading nose2-0.15.1-py3-none-any.whl.metadata (7.3 kB)
3.613 Collecting p4p (from -r p4p_reqs.txt (line 4))
3.624 Downloading p4p-4.1.12.tar.gz (110 kB)
3.649 Installing build dependencies: started
60.98 Installing build dependencies: finished with status 'done'
60.98 Getting requirements to build wheel: started
62.21 Getting requirements to build wheel: finished with status 'done'
62.21 Preparing metadata (pyproject.toml): started
62.54 Preparing metadata (pyproject.toml): finished with status 'done'
62.54 Requirement already satisfied: setuptools in /usr/local/lib/python3.10/site-packages (from epicscorelibs->-r p4p_reqs.txt (line 1)) (72.2.0)
62.54 Requirement already satisfied: setuptools-dso>=2.9a1 in /usr/local/lib/python3.10/site-packages (from epicscorelibs->-r p4p_reqs.txt (line 1)) (2.10)
62.54 Requirement already satisfied: numpy in /usr/local/lib/python3.10/site-packages (from epicscorelibs->-r p4p_reqs.txt (line 1)) (1.22.4)
62.54 Requirement already satisfied: pandas in /usr/local/lib/python3.10/site-packages (from meme@ git+https://github.com/slaclab/meme@af07afe431fa4503664aca8d9f83f786f85abbf0->-r p4p_reqs.txt (line 2)) (2.2.2)
62.55 Collecting epicscorelibs (from -r p4p_reqs.txt (line 1))
62.55 Downloading epicscorelibs-7.0.7.99.1.1a2.tar.gz (1.6 MB)
62.62 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 25.7 MB/s eta 0:00:00
62.75 Installing build dependencies: started
63.59 Installing build dependencies: finished with status 'done'
63.59 Getting requirements to build wheel: started
63.73 Getting requirements to build wheel: finished with status 'done'
63.73 Preparing metadata (pyproject.toml): started
63.87 Preparing metadata (pyproject.toml): finished with status 'done'
63.91 Collecting pvxslibs<1.4.0a1,>=1.3.1 (from p4p->-r p4p_reqs.txt (line 4))
63.92 Downloading pvxslibs-1.3.1.tar.gz (660 kB)
63.95 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 660.3/660.3 kB 27.8 MB/s eta 0:00:00
63.99 Installing build dependencies: started
87.24 Installing build dependencies: finished with status 'done'
87.24 Getting requirements to build wheel: started
87.34 Getting requirements to build wheel: finished with status 'done'
87.34 Preparing metadata (pyproject.toml): started
87.44 Preparing metadata (pyproject.toml): finished with status 'done'
87.53 Collecting numpy (from epicscorelibs->-r p4p_reqs.txt (line 1))
87.53 Downloading numpy-2.0.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.metadata (62 kB)
87.54 Requirement already satisfied: ply in /usr/local/lib/python3.10/site-packages (from p4p->-r p4p_reqs.txt (line 4)) (3.11)
87.56 Collecting setuptools-dso>=2.11a2 (from epicscorelibs->-r p4p_reqs.txt (line 1))
87.56 Downloading setuptools_dso-2.11a2-py2.py3-none-any.whl.metadata (1.6 kB)
87.58 Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/site-packages (from pandas->meme@ git+https://github.com/slaclab/meme@af07afe431fa4503664aca8d9f83f786f85abbf0->-r p4p_reqs.txt (line 2)) (2.9.0.post0)
87.58 Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/site-packages (from pandas->meme@ git+https://github.com/slaclab/meme@af07afe431fa4503664aca8d9f83f786f85abbf0->-r p4p_reqs.txt (line 2)) (2024.1)
87.58 Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.10/site-packages (from pandas->meme@ git+https://github.com/slaclab/meme@af07afe431fa4503664aca8d9f83f786f85abbf0->-r p4p_reqs.txt (line 2)) (2024.1)
87.58 Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/site-packages (from python-dateutil>=2.8.2->pandas->meme@ git+https://github.com/slaclab/meme@af07afe431fa4503664aca8d9f83f786f85abbf0->-r p4p_reqs.txt (line 2)) (1.16.0)
87.59 Downloading nose2-0.15.1-py3-none-any.whl (211 kB)
87.60 Downloading numpy-2.0.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (13.9 MB)
88.05 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.9/13.9 MB 31.6 MB/s eta 0:00:00
88.06 Downloading setuptools_dso-2.11a2-py2.py3-none-any.whl (23 kB)
88.09 Building wheels for collected packages: meme, p4p, epicscorelibs, pvxslibs
88.09 Building wheel for meme (setup.py): started
88.21 Building wheel for meme (setup.py): finished with status 'done'
88.21 Created wheel for meme: filename=meme-0.7.0-py3-none-any.whl size=17432 sha256=6a358a00f3e92a0a08ab9fe8c5311d109ba573690de60d3d5872534cf6308afd
88.21 Stored in directory: /tmp/pip-ephem-wheel-cache-p78zn6r9/wheels/09/cf/97/1bd7c844fc10fb8547d0072a9008a65cfaacf2858ff43489cd
88.21 Building wheel for p4p (pyproject.toml): started
92.49 Building wheel for p4p (pyproject.toml): finished with status 'error'
92.49 error: subprocess-exited-with-error
92.49
92.49 × Building wheel for p4p (pyproject.toml) did not run successfully.
92.49 │ exit code: 1
92.49 ╰─> [88 lines of output]
92.49 <string>:11: DeprecationWarning:
92.49
92.49 `numpy.distutils` is deprecated since NumPy 1.23.0, as a result
92.49 of the deprecation of `distutils` itself. It will be removed for
92.49 Python >= 3.12. For older Python versions it will remain present.
92.49 It is recommended to use `setuptools < 60.0` for those Python versions.
92.49 For more details, see:
92.49 https://numpy.org/devdocs/reference/distutils_status_migration.html
92.49
92.49
92.49 running bdist_wheel
92.49 running build
92.49 running build_py
92.49 creating build
92.49 creating build/lib.linux-aarch64-cpython-310
92.49 creating build/lib.linux-aarch64-cpython-310/p4p
92.49 copying src/p4p/gw.py -> build/lib.linux-aarch64-cpython-310/p4p
92.49 copying src/p4p/disect.py -> build/lib.linux-aarch64-cpython-310/p4p
92.49 copying src/p4p/__init__.py -> build/lib.linux-aarch64-cpython-310/p4p
92.49 copying src/p4p/rpc.py -> build/lib.linux-aarch64-cpython-310/p4p
92.49 copying src/p4p/util.py -> build/lib.linux-aarch64-cpython-310/p4p
92.49 copying src/p4p/version.py -> build/lib.linux-aarch64-cpython-310/p4p
92.49 copying src/p4p/wrapper.py -> build/lib.linux-aarch64-cpython-310/p4p
92.49 creating build/lib.linux-aarch64-cpython-310/p4p/nt
92.49 copying src/p4p/nt/__init__.py -> build/lib.linux-aarch64-cpython-310/p4p/nt
92.49 copying src/p4p/nt/ndarray.py -> build/lib.linux-aarch64-cpython-310/p4p/nt
92.49 copying src/p4p/nt/scalar.py -> build/lib.linux-aarch64-cpython-310/p4p/nt
92.49 copying src/p4p/nt/common.py -> build/lib.linux-aarch64-cpython-310/p4p/nt
92.49 copying src/p4p/nt/enum.py -> build/lib.linux-aarch64-cpython-310/p4p/nt
92.49 creating build/lib.linux-aarch64-cpython-310/p4p/client
92.49 copying src/p4p/client/__init__.py -> build/lib.linux-aarch64-cpython-310/p4p/client
92.49 copying src/p4p/client/cli.py -> build/lib.linux-aarch64-cpython-310/p4p/client
92.49 copying src/p4p/client/thread.py -> build/lib.linux-aarch64-cpython-310/p4p/client
92.49 copying src/p4p/client/Qt.py -> build/lib.linux-aarch64-cpython-310/p4p/client
92.49 copying src/p4p/client/asyncio.py -> build/lib.linux-aarch64-cpython-310/p4p/client
92.49 copying src/p4p/client/cothread.py -> build/lib.linux-aarch64-cpython-310/p4p/client
92.49 copying src/p4p/client/raw.py -> build/lib.linux-aarch64-cpython-310/p4p/client
92.49 creating build/lib.linux-aarch64-cpython-310/p4p/test
92.49 copying src/p4p/test/test_nt.py -> build/lib.linux-aarch64-cpython-310/p4p/test
92.49 copying src/p4p/test/test_asyncio.py -> build/lib.linux-aarch64-cpython-310/p4p/test
92.49 copying src/p4p/test/asynciotest.py -> build/lib.linux-aarch64-cpython-310/p4p/test
92.49 copying src/p4p/test/__init__.py -> build/lib.linux-aarch64-cpython-310/p4p/test
92.49 copying src/p4p/test/qttest.py -> build/lib.linux-aarch64-cpython-310/p4p/test
92.49 copying src/p4p/test/test_server.py -> build/lib.linux-aarch64-cpython-310/p4p/test
92.49 copying src/p4p/test/test_sharedpv.py -> build/lib.linux-aarch64-cpython-310/p4p/test
92.49 copying src/p4p/test/test_rpc.py -> build/lib.linux-aarch64-cpython-310/p4p/test
92.49 copying src/p4p/test/test_type.py -> build/lib.linux-aarch64-cpython-310/p4p/test
92.49 copying src/p4p/test/test_gw.py -> build/lib.linux-aarch64-cpython-310/p4p/test
92.49 copying src/p4p/test/test_asLib.py -> build/lib.linux-aarch64-cpython-310/p4p/test
92.49 copying src/p4p/test/test_cothread.py -> build/lib.linux-aarch64-cpython-310/p4p/test
92.49 copying src/p4p/test/cothreadtest.py -> build/lib.linux-aarch64-cpython-310/p4p/test
92.49 copying src/p4p/test/test_value.py -> build/lib.linux-aarch64-cpython-310/p4p/test
92.49 copying src/p4p/test/test_client_raw.py -> build/lib.linux-aarch64-cpython-310/p4p/test
92.49 copying src/p4p/test/utils.py -> build/lib.linux-aarch64-cpython-310/p4p/test
92.49 copying src/p4p/test/test_qt.py -> build/lib.linux-aarch64-cpython-310/p4p/test
92.49 copying src/p4p/test/test_client_thread.py -> build/lib.linux-aarch64-cpython-310/p4p/test
92.49 creating build/lib.linux-aarch64-cpython-310/p4p/server
92.49 copying src/p4p/server/__init__.py -> build/lib.linux-aarch64-cpython-310/p4p/server
92.49 copying src/p4p/server/cli.py -> build/lib.linux-aarch64-cpython-310/p4p/server
92.49 copying src/p4p/server/thread.py -> build/lib.linux-aarch64-cpython-310/p4p/server
92.49 copying src/p4p/server/asyncio.py -> build/lib.linux-aarch64-cpython-310/p4p/server
92.49 copying src/p4p/server/cothread.py -> build/lib.linux-aarch64-cpython-310/p4p/server
92.49 copying src/p4p/server/raw.py -> build/lib.linux-aarch64-cpython-310/p4p/server
92.49 creating build/lib.linux-aarch64-cpython-310/p4p/asLib
92.49 copying src/p4p/asLib/lex.py -> build/lib.linux-aarch64-cpython-310/p4p/asLib
92.49 copying src/p4p/asLib/yacc.py -> build/lib.linux-aarch64-cpython-310/p4p/asLib
92.49 copying src/p4p/asLib/__init__.py -> build/lib.linux-aarch64-cpython-310/p4p/asLib
92.49 copying src/p4p/asLib/pvlist.py -> build/lib.linux-aarch64-cpython-310/p4p/asLib
92.49 copying src/p4p/example.conf -> build/lib.linux-aarch64-cpython-310/p4p
92.49 copying src/p4p/pvagw@.service -> build/lib.linux-aarch64-cpython-310/p4p
92.49 running build_ext
92.49 building 'p4p._p4p' extension
92.49 Warning: Can't read registry to find the necessary compiler setting
92.49 Make sure that Python modules winreg, win32api or win32con are installed.
92.49 INFO: C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC
92.49
92.49 creating build/temp.linux-aarch64-cpython-310
92.49 creating build/temp.linux-aarch64-cpython-310/src
92.49 creating build/temp.linux-aarch64-cpython-310/src/p4p
92.49 INFO: compile options: '-D_GNU_SOURCE -D_DEFAULT_SOURCE -Dlinux -DUNIX -D_GLIBCXX_USE_CXX11_ABI=1 -D__PYX_EXTERN_C=extern -DPY_ARRAY_UNIQUE_SYMBOL=PVXS_PyArray_API -DPVXS_ENABLE_EXPERT_API -I/tmp/pip-build-env-yhvusxpq/overlay/lib/python3.10/site-packages/numpy/_core/include -I/tmp/pip-build-env-yhvusxpq/overlay/lib/python3.10/site-packages/epicscorelibs/include -I/tmp/pip-build-env-yhvusxpq/overlay/lib/python3.10/site-packages/pvxslibs/include -Isrc -Isrc/p4p -I/usr/local/include/python3.10 -c'
92.49 extra options: '-g0 -std=c++11'
92.49 INFO: gcc: src/p4p/_p4p.cpp
92.49 INFO: gcc: src/pvxs_client.cpp
92.49 INFO: gcc: src/pvxs_sharedpv.cpp
92.49 INFO: gcc: src/pvxs_source.cpp
92.49 INFO: gcc: src/pvxs_type.cpp
92.49 INFO: gcc: src/pvxs_value.cpp
92.49 error: Command "gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -D_GNU_SOURCE -D_DEFAULT_SOURCE -Dlinux -DUNIX -D_GLIBCXX_USE_CXX11_ABI=1 -D__PYX_EXTERN_C=extern -DPY_ARRAY_UNIQUE_SYMBOL=PVXS_PyArray_API -DPVXS_ENABLE_EXPERT_API -I/tmp/pip-build-env-yhvusxpq/overlay/lib/python3.10/site-packages/numpy/_core/include -I/tmp/pip-build-env-yhvusxpq/overlay/lib/python3.10/site-packages/epicscorelibs/include -I/tmp/pip-build-env-yhvusxpq/overlay/lib/python3.10/site-packages/pvxslibs/include -Isrc -Isrc/p4p -I/usr/local/include/python3.10 -c src/pvxs_value.cpp -o build/temp.linux-aarch64-cpython-310/src/pvxs_value.o -g0 -std=c++11" failed with exit status 1
92.49 [end of output]
92.49
92.49 note: This error originates from a subprocess, and is likely not a problem with pip.
92.49 ERROR: Failed building wheel for p4p
92.49 Building wheel for epicscorelibs (pyproject.toml): started
111.7 Building wheel for epicscorelibs (pyproject.toml): finished with status 'done'
111.7 Created wheel for epicscorelibs: filename=epicscorelibs-7.0.7.99.1.1a2-cp310-cp310-linux_aarch64.whl size=5074916 sha256=48c1a2e610bb9c1aaf55ef8479f4e1d1a8d126e778fa68fdf33b22764b25d72c
111.7 Stored in directory: /tmp/pip-ephem-wheel-cache-p78zn6r9/wheels/d4/8b/22/f36a826f9e51101328b7af1e32bcdc10a020bb4adf23564851
111.7 Building wheel for pvxslibs (pyproject.toml): started
121.4 Building wheel for pvxslibs (pyproject.toml): finished with status 'done'
121.4 Created wheel for pvxslibs: filename=pvxslibs-1.3.1-cp310-cp310-linux_aarch64.whl size=2441106 sha256=96afddbe0de298287a8d9fb81bdb36eb56f4de595ba65fa97fc80e72149c8f4b
121.4 Stored in directory: /tmp/pip-ephem-wheel-cache-p78zn6r9/wheels/7e/f4/76/05f2f0bc6dfafa9355714578b7683c6e38a4ea4c4fb9bcd7ca
121.4 Successfully built meme epicscorelibs pvxslibs
121.4 Failed to build p4p
I've tried cloning the p4p repo and adjusting the contents of setup.py and pyproject.toml, but then I run into incompatibility issues with pvxslibs. I don't know how necessary those required dependencies are, so I don't want to just bulldoze through them all.
finnoshea commented
At the risk of commenting too much, when I change pyproject.toml to be
[build-system]
requires = [
"setuptools",
"setuptools_dso>=2.7a1,<=2.10; python_version<='3.11'",
"setuptools_dso>=2.11a2; python_version>='3.12'",
"wheel",
"numpy",
#"numpy>=2.0.1; python_version>='3.10'",
"numpy<1.23; python_version>='3.10'",
"Cython>=0.20",
#"epicscorelibs==7.0.7.99.0.2; python_version<='3.11'",
#"epicscorelibs>=7.0.7.99.1.1a2; python_version>='3.12'",
"pvxslibs==1.3.1; python_version<='3.11'",
"pvxslibs>=1.3.2a1; python_version>='3.12'",
]
in a cloned repo, I get the following error:
0.202 Processing /opt/p4p
0.203 Installing build dependencies: started
25.19 Installing build dependencies: finished with status 'error'
25.19 error: subprocess-exited-with-error
25.19
25.19 × pip subprocess to install build dependencies did not run successfully.
25.19 │ exit code: 1
25.19 ╰─> [44 lines of output]
25.19 Ignoring setuptools_dso: markers 'python_version >= "3.12"' don't match your environment
25.19 Ignoring pvxslibs: markers 'python_version >= "3.12"' don't match your environment
25.19 Collecting setuptools
25.19 Downloading setuptools-72.2.0-py3-none-any.whl.metadata (6.6 kB)
25.19 Collecting setuptools_dso<=2.10,>=2.7a1
25.19 Downloading setuptools_dso-2.10-py2.py3-none-any.whl.metadata (1.6 kB)
25.19 Collecting wheel
25.19 Downloading wheel-0.44.0-py3-none-any.whl.metadata (2.3 kB)
25.19 Collecting numpy
25.19 Downloading numpy-2.0.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.metadata (62 kB)
25.19 Downloading numpy-1.22.4-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.metadata (2.0 kB)
25.19 Collecting Cython>=0.20
25.19 Downloading Cython-3.0.11-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.metadata (3.2 kB)
25.19 Collecting pvxslibs==1.3.1
25.19 Downloading pvxslibs-1.3.1.tar.gz (660 kB)
25.19 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 660.3/660.3 kB 28.9 MB/s eta 0:00:00
25.19 Installing build dependencies: started
25.19 Installing build dependencies: finished with status 'done'
25.19 Getting requirements to build wheel: started
25.19 Getting requirements to build wheel: finished with status 'done'
25.19 Preparing metadata (pyproject.toml): started
25.19 Preparing metadata (pyproject.toml): finished with status 'done'
25.19 Collecting epicscorelibs<7.0.7.99.2,>=7.0.7.99.1.1a2 (from pvxslibs==1.3.1)
25.19 Downloading epicscorelibs-7.0.7.99.1.1a2.tar.gz (1.6 MB)
25.19 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 29.8 MB/s eta 0:00:00
25.19 Installing build dependencies: started
25.19 Installing build dependencies: finished with status 'done'
25.19 Getting requirements to build wheel: started
25.19 Getting requirements to build wheel: finished with status 'done'
25.19 Preparing metadata (pyproject.toml): started
25.19 Preparing metadata (pyproject.toml): finished with status 'done'
25.19 INFO: pip is looking at multiple versions of epicscorelibs to determine which version is compatible with other requirements. This could take a while.
25.19 ERROR: Cannot install pvxslibs, pvxslibs==1.3.1 and setuptools_dso<=2.10 and >=2.7a1 because these package versions have conflicting dependencies.
25.19
25.19 The conflict is caused by:
25.19 The user requested setuptools_dso<=2.10 and >=2.7a1
25.19 pvxslibs 1.3.1 depends on setuptools-dso>=2.7a1
25.19 epicscorelibs 7.0.7.99.1.1a2 depends on setuptools-dso>=2.11a2
25.19
25.19 To fix this you could try to:
25.19 1. loosen the range of package versions you've specified
25.19 2. remove package versions to allow pip to attempt to solve the dependency conflict
25.19
25.19 ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
25.19 [end of output]
25.19
25.19 note: This error originates from a subprocess, and is likely not a problem with pip.
25.19 error: subprocess-exited-with-error
25.19
25.19 × pip subprocess to install build dependencies did not run successfully.
25.19 │ exit code: 1
25.19 ╰─> See above for output.
25.19
25.19 note: This error originates from a subprocess, and is likely not a problem with pip.
mdavidsaver commented
Apologies for the verbose dump ...
This verbosity can be avoided by attaching as a text file.