Ask about RuntimeError: Graph disconnected
Opened this issue · 0 comments
Dear Shibuiwilliam,
Thank you so much for your github about mixture of experts for keras.
I learnt a lot from them.
I have a problem related to "Graph disconnected" when I try to use them for my system.
- I have 6 experts that are 6 different parallel CNN structures, and using the same input. The input of experts are a pair of 4D arrays, in which size of them is (None, 128,50,1) and each expert includes Input layer for the first layer of a model and returning of each model is model=Model([input1,input2], output) - these are differences to your expert structures.
- I get output of these experts: e0, e1, e2, e3, e4, e5 by calling "model.output"
- I define a gating model including Input layer like other experts and followed layers like yours (dense, dropout, reshape)
- The below code is my MoE_output and MoE_model.
'''
from keras.layers import Lambda
MoE_output = Lambda(lambda gx: (gx[0]*gx[6][:,:,0]) + (gx[1]*gx[6][:,:,1])+
(gx[2]*gx[6][:,:,2]) + (gx[3]*gx[6][:,:,3])+
(gx[4]*gx[6][:,:,4]) + (gx[5]*gx[6][:,:,5])
, output_shape=(10,))([e0, e1, e2, e3, e4, e5, gating_model.output])
X1_tmp = Input(shape=(X1_train.shape[1:]))
X2_tmp = Input(shape=(X2_train.shape[1:]))
MoE_model = Model(inputs=[X1_tmp, X2_tmp], outputs=MoE_output)
'''
--->If I do not use Input layers for these array inputs, they will cause a error" unhashbleerror: numpy.array"
since in topology.py of keras, there is an unsatified condition "len(set(seft.inputs))"!= len(self.inputs)"
so I change these array inputs to tensors by using Input layer of keras.
-->If I use Input layers and then build MoE_model as above, it will launch graphic disconnected error.
Althought I check Input of experts, they are same as <Tensor("input_2_6:0", shape=(?, 128, 50, 1), dtype=float32) >
'''
*** RuntimeError: Graph disconnected: cannot obtain value for tensor Tensor("input_2_6:0", shape=(?, 128, 50, 1), dtype=float32) at layer "input_2". The following previous layers were accessed without issue: []
'''
Please give me some advise to solve this problem.
Looking forward to hearing from you soon
Thank you so much,
Truc