HOW TO DO THE FUSION

[saadbenda]

Hi guys, thank you for your incredible work. I want to know how exactly and precisly you do the fusion between the rnn and the cnn. I want to use that same logic. If you can send me some code it will be even better. Thank You Again.

[shuidongliu]

Hi, you can use the following functions to get the fusion. va_rnn_results and va_cnn_results are the predictions of last fc layer before softmax layer.

def softmax(data):
e = np.exp(data - np.amax(data, axis=-1, keepdims=True))
s = np.sum(e, axis=-1, keepdims=True)
return e / s

def acc(va_rnn_results, va_cnn_results, label):
pred = va_rnn_results + 4 * va_cnn_results
pred_label = np.argmax(softmax(pred), axis= -1)
total = ((label-pred_label)==0).sum()

return float(total) / label.shape[0] * 100