Error with older numpy
Closed this issue · 8 comments
Using numpy==1.18.5 resulted in the following error
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
Traceback (most recent call last):
[...]
import digital_rf as drf
File "/home/danielk/venvs/sorts/lib/python3.7/site-packages/digital_rf/__init__.py", line 4, in <module>
from .digital_rf_hdf5 import *
File "/home/danielk/venvs/sorts/lib/python3.7/site-packages/digital_rf/digital_rf_hdf5.py", line 36, in <module>
from . import _py_rf_write_hdf5, digital_metadata, list_drf
ImportError: numpy.core.multiarray failed to import
and upgrading to numpy==1.20.3 fixed the problem.
Yeah, you generally can't use a numpy that's older than what it was compiled with. Did you install with pip? What platform? Hopefully I can sort this out by fixing the PyPI package.
This was on a Ubuntu 16 LTS (I desperately need to update that machine when I get time, but that should be independent of this issue).
I installed with pip install digital_rf, but numpy already existed in my virtualenv due to another package being installed prior that listed an old numpy version as a dependency. Hence, I assume that the setup.py ( https://github.com/MITHaystack/digital_rf/blob/master/python/setup.py#L212 ) saw the existing version and did not update it or state a conflict.
So yeah, I think that a simple update of the that line and reupload to PyPI would fix it?
It's not the setup.py per-se, because for people installing from source it doesn't matter what version of numpy is used. I use a particular setup (https://github.com/ryanvolz/digital_rf-wheels) to compile the packages for upload to PyPI, and I usually try to use the oldest numpy available to avoid this problem. I'll look to see why that didn't happen in this case and fix it.
It should have been built against numpy=1.14, so I'm still not sure what happened and why it's broken. These wheels for PyPI are a pain to set up and I haven't done anything more than plug in new versions over the past couple years, so perhaps it is time to re-assess the tooling and see if a better method exists now.
Oh I see! It was quite some time since I setup a automatic packing pipeline so Ill just say good luck!
Only thing that might be of use is that I once had the problem of the package not having the same requirement in the core meta data as was used in the tests, which created a whole lot of unnecessary debugging.
It seems like pip has gone and done something different from earlier versions that silently uses the newest numpy version for building: pypa/pip#9542. Apparently anyone building wheels with pip now by default has no control over what versions of the dependencies are used; only what's specified in pyproject.toml matters, unless you turn off build isolation. So the proper thing to do from their perspective is for me to use oldest-supported-numpy in pyproject.toml, which I can absolutely do but will require a new release. I guess the workaround would be to turn off build isolation for the wheel builds. I'll probably try to put together a new release soon.
I have uploaded new wheels for digital_rf==2.6.6 (with build number 1) that should work with the oldest available numpy wheels.
Oh boy, good job catching that. Thanks! (will keep a mental note for the next time I need to build wheels)