Emergent-Behaviors-in-Biology/community-simulator

Parallelization option is an infinite loop

Opened this issue · 0 comments

It looks like the Propagate function turns into an infinite loop if parallelize is specified. Below is a script to reproduce this issue.
.

import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
from community_simulator.usertools import (MakeConsumerDynamics, MakeResourceDynamics,  MakeParams,
                                           MakeMatrices, MakeInitialState, a_default)
from community_simulator import Community
from community_simulator.visualization import PlotTraj, StackPlot
import pickle
np.random.seed(1)
assumptions=a_default.copy() #Start with default parameters
assumptions['n_wells'] = 100 #Add more wells
assumptions['l'] = 0.8 #Assign a high leakage fraction to allow for a lot of crossfeeding
R0_food = np.linspace(30,300,assumptions['n_wells']) #Make a gradient of food supply levels
assumptions['R0_food'] = R0_food
assumptions['SA'] = 2 * np.ones(2)
assumptions['Sgen'] = 1
assumptions['S'] = 5
mp = {
    'regulation' : 'independent',
    'supply': 'external',
    'response': 'type I'
}
def dNdt(N, R, params):
    return MakeConsumerDynamics(mp)(N, R, params)
def dRdt(N, R, params):
    return MakeResourceDynamics(mp)(N, R, params)
dynamics = [dNdt, dRdt]
init_state = MakeInitialState(assumptions)
params = MakeParams(assumptions)

test_colony2 = Community(init_state,dynamics,params,parallel=True)
test_colony2.Propagate(T=5)

This will take hours to run without terminating - below a traceback after interrupting; it looks like there are processing that perpetually waiting

---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
<ipython-input-8-5e13682f49db> in <module>
      1 test_colony2 = Community(init_state,dynamics,params,parallel=True)
----> 2 test_colony2.Propagate(T=5)

~/Documents/research/community-simulator/community_simulator/__init__.py in Propagate(self, T, compress_resources, compress_species)
    260         if self.parallel:
    261             pool = Pool()
--> 262             y_out = np.asarray(pool.map(IntegrateTheseWells,well_info)).squeeze().T
    263             pool.close()
    264         else:

~/miniconda3/envs/catvae/lib/python3.8/multiprocessing/pool.py in map(self, func, iterable, chunksize)
    362         in a list that is returned.
    363         '''
--> 364         return self._map_async(func, iterable, mapstar, chunksize).get()
    365 
    366     def starmap(self, func, iterable, chunksize=None):

~/miniconda3/envs/catvae/lib/python3.8/multiprocessing/pool.py in get(self, timeout)
    763 
    764     def get(self, timeout=None):
--> 765         self.wait(timeout)
    766         if not self.ready():
    767             raise TimeoutError

~/miniconda3/envs/catvae/lib/python3.8/multiprocessing/pool.py in wait(self, timeout)
    760 
    761     def wait(self, timeout=None):
--> 762         self._event.wait(timeout)
    763 
    764     def get(self, timeout=None):

~/miniconda3/envs/catvae/lib/python3.8/threading.py in wait(self, timeout)
    556             signaled = self._flag
    557             if not signaled:
--> 558                 signaled = self._cond.wait(timeout)
    559             return signaled
    560 

~/miniconda3/envs/catvae/lib/python3.8/threading.py in wait(self, timeout)
    300         try:    # restore state no matter what (e.g., KeyboardInterrupt)
    301             if timeout is None:
--> 302                 waiter.acquire()
    303                 gotit = True
    304             else:

KeyboardInterrupt: 

Interesting, if a I specify the parallelize option to be false, it runs in a second, utilizing all of the cores on my machine.
Raising here in case someone runs into similar issues.