KaveIO/PhiK

NCORES Related Crash on Windows

IronFarm opened this issue · 2 comments

Hi,

It's not totally clear to me why but this line causes a crash on a Windows machine with 64 or more cores:

phik_list = Parallel(n_jobs=NCORES)(delayed(_calc_phik)(co, data_binned[list(co)], noise_correction)

I believe it is related to this long-present bug in Python:
https://bugs.python.org/issue26903

I'd suggest allowing the NCORES to be overridden with an environment variable or manually decreasing the value when it is >=64 on a Windows machine.

Thanks

mbaak commented

Working on it.

mbaak commented

phik v0.9.12 should work again now on Windows machine with 64+ cores. Can you give it a try?

The number of cores used can now also be configured. Eg. for 4 cores:
import phik.config
phik.config.ncores = 4

... before you call phik.