System shut down with `HawkesEM` and `multiprocessing`
Opened this issue · 0 comments
rambor12 commented
Hello,
During the Monte Carlo simulation, I found that the HawkesEM
with multiprocessing
using apply_async
leads to a sudden system shut down.
This is still happening with another parallel processing package ray
.
Since I am not an expert in software engineering, I have no idea how to handle this problem.
So I just ask here to get any hint to solve this issue.
Here is the example:
import gc
import numpy as np
from tick.hawkes import SimuHawkesExpKernels, HawkesExpKern, HawkesEM
def run(seed):
# Point generation
hawkes = SimuHawkesExpKernels([[alpha/beta]], [[beta]], baseline,
end_time=T, verbose=False, seed=seed,
period_length=T)
hawkes.simulate()
points = hawkes.timestamps[0]
del hawkes
# Estimation
model = HawkesEM(kernel_support=10)
model.fit([[points]])
res = model.kernel.sum()
del points, model
gc.collect()
print(end='.')
return np.array(res)
N_sim = 6000
alpha = 1.6
beta = 2.0
T = 40000.
baseline = np.repeat(1,T).reshape((1,-1))
from functools import partial
from multiprocessing import Pool, cpu_count
def log_out(i, out):
pool_out[i] = out
pool_out = {}
pool = Pool(cpu_count()-2)
try:
for seed in range(N_sim):
if seed not in pool_out:
pool.apply_async(run, args=(seed,),
callback=partial(log_out, seed))
pool.close()
pool.join()
except KeyboardInterrupt:
pool.terminate()
pool_out=dict(sorted(pool_out.items()))
The run tine would be several minutes depends on your system.
Here is my system info:
Linux-5.4.0-107-generic-x86_64-with-debian-buster-sid
Python: 3.7.3 (default, Mar 27 2019, 22:11:17)
[GCC 7.3.0]
Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz
tick Version: 0.6.0.0
Thank you for the reading.