`evaluate_cut` running slow ?
Closed this issue · 4 comments
Hi @wmayner,
can you give some advice/help why I find it takes much longer to run compute.evaluate_cut
compared to simply running compute.big_mip
?
Here's a minimal script you can run, and also a net you can download
https://www.dropbox.com/s/705c2ne4gpj13qo/network_4_nodes_dev_pyphi.json?dl=0
to reproduce it exactly, (but I get the same behaviour for all other subsystems
too).
import pyphi
from pyphi.compute.big_phi import evaluate_cut
import json
from random import randint
from time import time
##---------------------------------------------------
# Import net and setup system
##---------------------------------------------------
import json
path_to_saved_net = "/home/ajay/Downloads/network.json"
with open(path_to_saved_net) as data_file:
data = json.load(data_file)
tpm = data['tpm']
cm = data['cm']
state = data['state']
network = pyphi.Network(tpm, connectivity_matrix=cm)
subsystem = pyphi.Subsystem(network, state, range(network.size))
#subsystem.node_indices
pyphi.config.CACHE_BIGMIPS = False
pyphi.config.PARALLEL_CUT_EVALUATION = False
##---------------------------------------------------
# Simple Big mip calculation
##---------------------------------------------------
def compute_big_mip(subsystem):
start = time()
for i in range (0,100):
mip = pyphi.compute.big_mip( subsystem )
print(time()-start)
##---------------------------------------------------
# Simple Big mip calculation and a random cut
##---------------------------------------------------
unpartitioned_constellation = pyphi.compute.constellation(subsystem)
all_bipartitions = pyphi.utils.directed_bipartition( subsystem.node_indices )[1:-1] # can be precomputed when the environment is initialized
num_all_directed_bipartitions = len(all_bipartitions)
def compute_big_mip_and_a_random_cut( subsystem ):
start = time()
for t in range (0,100):
mip = pyphi.compute.big_mip(subsystem)
i = randint(0, num_all_directed_bipartitions-1)
rand_bipartition = all_bipartitions[ i : i+1 ] # random directed bipartition
random_cut = [ pyphi.models.cuts.Cut ( bipartition[0], bipartition[1] ) for bipartition in rand_bipartition ] # random_cut
mip_of_rand_cut = evaluate_cut(subsystem, random_cut[0], unpartitioned_constellation) # pyphi/compute/big_phi.py#L27
print(time()-start)
##---------------------------------------------------
# Compare timings
##---------------------------------------------------
>>> compute_big_mip(subsystem)
10.356880187988281
>>>
>>> compute_big_mip_and_a_random_cut( subsystem )
39.57922887802124
So my question is why does it take 3 times as long to evaluate a single random cut, compare to evaluating the big_mip
for a full system which requires searching over many cuts?
If I can sort this out I'll be able to run some experiments, and can show you how to apply ML here.
Thanks a lot for your help,
Aj
Update, I think this is due to the way pyphi catches things, (but I don't know how to turn this on and off yet)?
I generated and stored 100, random tpm
s and state
s, and then ran the above on them, and got much more common sense times,
>>> compute_big_mip()
33.894142627716064
>>> compute_big_mip_and_a_random_cut()
31.76900362968445
So I'll close this, as it seems like it's really use case confusion, rather than problem with pyphi
Oh dear maybe I spoke too soon? Now if I calculate the random cut only, it still seems very slow,
>>> def compute_a_random_cut_only():
... start = time()
... for t in range (0,100):
... network = pyphi.Network(tpms[t], connectivity_matrix=cms[t])
... subsystem = pyphi.Subsystem(network, states[t], range(network.size))
... #mip = pyphi.compute.big_mip(subsystem)
... i = randint(0, num_all_directed_bipartitions-1)
... rand_bipartition = all_bipartitions[ i : i+1 ] # random directed bipartition
... random_cut = [ pyphi.models.cuts.Cut ( bipartition[0], bipartition[1] ) for bipartition in rand_bipartition ] # random_cut
... unpartitioned_constellation = pyphi.compute.constellation(subsystem)
... mip_of_rand_cut = evaluate_cut(subsystem, random_cut[0], unpartitioned_constellation) # pyphi/compute/big_phi.py#L27
... print(time()-start)
...
>>> compute_a_random_cut_only()
30.439067840576172
I would have expected it to be much faster if it's only got to calculate one random cut, and not the full subsystem's big_mip
. Any ideas where all the computation time is going?
Hi Ajay,
I finally had a chance to look into this, and I can't seem to reproduce it. I wrote a similar script, and I see results more in line with what we'd expect:
Timing functions (4 nodes, 100 iterations)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
big_mip: done.
Mean time: 0.6552s
naive_big_mip: done.
Mean time: 0.7885s
unpartitioned_constellation: done.
Mean time: 0.0965s
random_cut: done.
Mean time: 0.0680s
When the script is run with 5 nodes rather than 4, I get similar results:
Timing functions (4 nodes, 100 iterations)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
big_mip: done.
Mean time: 31.5975s
naive_big_mip: done.
Mean time: 32.6524s
unpartitioned_constellation: done.
Mean time: 2.0442s
random_cut: done.
Mean time: 0.9371s
Hi Will, @wmayner,
sorry for the late reply - I'm not ignoring you - I got caught up in something else. I'm looking into this now, and going to reply to your email soon.
It's great that you've got the pyphi-users forum rolling - maybe you want to put a link to it on the front page here to let new users know it exists?
Best,
Aj