python-hydro/pyro2

lm_atm-bubble regression test fails with numpy 1.24

Closed this issue · 5 comments

yut23 commented

First showed up in https://github.com/python-hydro/pyro2/actions/runs/4865084901.

I can reproduce the failure on the current main branch with python 3.11, numpy 1.24.3, and numba 0.57.0, but it works fine if I downgrade to numpy 1.23.5, leaving everything else the same.

I also modified the test to run for only one step and got similar results. However, when I ran this version under valgrind, it passed fine with numpy 1.24.

yut23 commented

Running with limiter = 0 seems to fix it too

zingale commented

should we pin the version to numpy < 1.24?

yut23 commented

That sounds like a good idea while we track down what changed in numpy

yut23 commented

I managed to track this down to numpy 1.24 now using SVML on Skylake processors for power and arctan: numpy/numpy#23289

Running with NPY_DISABLE_CPU_FEATURES="AVX512_SKX" gives the exact same output as the benchmark, for both numpy versions.

  • 👍1
zingale commented

for now, I'll disable this for the tests. But once 1.24 is more widely used, we should rerun the tests with this on.
We might also want to add a note to the docs