numpy 1.19.5 bug

[piperfw]

Using numpy ver 1.19.5 breaks the package on my system ('fatal Python Error: illegal instruction...'). This is not present at 1.19.4.
This is an issue as 1.19.5 is the last numpy version with Python 3.6 support, so will be installed by default when creating a new development environment in many cases.

Suggest replacing numpy>=1.18.0 with numpy>=1.18.0, <1.19.5 in requirements.txt until we move to a later Python version (#80).

[gefux]

I'd agree that this is probably a good quick-fix. Having such an restrictive dependency on such a commonly used package is not ideal - this probably tells us that we should move on soon to a newer version of Python as suggested in #80.

[gefux]

Actually, given that moving on to Python version 3.10 is not really much of an effort, it's probably better to do that (#80) instead.

[piperfw]

Great, well, it was one line so I've changed that. It's hard to say whether upgrading to 3.10 would be effortless without actually trying it. Regardless, this issue can be closed.

[gefux]

I cannot reconstruct that bug. Here is what I've tried:

$ git checkout 7445bce012a9011e91bb8de72ecafc362205a591
# This is the commit before the numpy version fix.
$ conda --version   # conda 22.9.0
$ conda create --name issue81 python=3.6.2
$ conda activate issue81
$ python3.6 -m pip install numpy==1.19.5
$ python3.6 -m pip install -r requirements_ci.txt
$ python3.6 -m pip list   # Still shows numpy==1.19.5
$ python3.6 -m pytest   # All tests pass!

@piperfw: Could you post a minimal failing example?

[gefux]

Actually, given that moving on to Python version 3.10 is not really much of an effort, it's probably better to do that (#80) instead.

I retract this comment above (see discussion in issue #80). If it doesn't take any significant amount of maintaining we should try to support the widest range of python and numpy versions, i.e. try to support python>=X and numpy>=Y with the lowest values of X and Y.

[piperfw]

Interesting, could you upload you pip list so we can compare? Mine after the following (failure) is attached.

pfw_piplist.txt

git checkout 7445bce012a9011e91bb8de72ecafc362205a591
python3.6 -m venv issue81 # python --version 3.6.15
source issue81/bin/activate
python3.6 -m pip install numpy==1.19.5
python3.6 -m pip install -r requirements_ci.txt
python3.6 -m pip list > pfw_piplist.txt # numpy==1.19.5
python3.6 -m pytest # Fatal Python error: Illegal instruction

The only difference not captured by the pip list I think is that I'm using python 3.6.15, as that is the only 3.6+ on my system and venv doesn't allow for the option to specify a different version as you do with conda. Perhaps the first thing to try would be your commands again, but setting python=3.6.15?

[gefux]

I don't have python 3.6.15 available but I used python 3.6.13 this time and the tests still passed. Here is the piplist
gefux_piplist.txt

[gefux]

I had trouble installing python 3.6.15 on my machine, so I used docker with the python:3.6.15 docker image and still had no problem with the tests. Here again the piplist
gefux_piplist_docker.txt

Have you tried un/reinstalling python 3.6.15 ?

[piperfw]

Okay, this appears to be some silent issue when building with an outdated version of pip :

• the only notable difference between our two version lists was pip 18.1 (mine) vs 21.2.4 (yours), with associated older versions of setuptools (40.6.2 vs 57.5.0) and wheel (0.37.0 vs 0.37.1) being installed when running pip install -r requirements_ci.txt
• Running pip install pip==21.2.4 (or a later version of pip) before installing the requirements, the issue is gone i.e. I can use numpy==1.19.5

I don't think it's worth the time of day trying to find which specific pip or setuptools/wheel version leads to this problem. I don't mind if you revert my commit on this or not (I'll just have to remember to always upgrade pip when installing a new environment).

[gefux]

Ok. I guess in this case this is a too specific and unlikely combination of installations that does not justify to restrict python or numpy versions more than we had previously. I will then go ahead an revert the commit if that's alright.

[gefux]

Done. Closing this now. Feel free to reopen if the issue reappears with a different combination of python and pip versions.