integer overflow in `linalg.sh_legendre` on windows
jorenham opened this issue · 0 comments
See: https://github.com/jorenham/Lmo/actions/runs/6687308385/job/18167801378
The np.int_
is a C long, which is 32bits in win64, because, well.. err... logic...?
... anyway, this is could be solved by replacing all np.int_
's by np.int64
.
Another option could be to switch _sh_jacobi_f
in the cases where _sh_jacobi_i
will overflow given k
and dtype
in
sh_legendre
, and similarly in sh_jacobi(k, a, b, dtype)
.
Note that in practise, this is only a problem in _lm._l_weights_pwm
. The _lm._l_weights_ostat
method is slower, slightly less precise, but a lot less likely to overflow.
For any users facing this issue; the workaround is to use e.g. trim=1e-15
instead of trim=0
. This forces it to use the ostat
weights, since the pwm
weights can only handle integer trimming.