SciSharp/TensorFlow.NET

Is bidirectional for LSTM even supported ? I am getting empty ( in C#/Tensorflow.NET 0 parameters) [Question]:

jaroslav-hook opened this issue · 8 comments

jaroslav-hook commented

Description

        ILayer forwardLayer = keras.layers.LSTM(units: 10, activation: keras.activations.Sigmoid, return_sequences: true, go_backwards: false);
        ILayer backwardlayer = keras.layers.LSTM(units: 10, activation: keras.activations.Sigmoid, return_sequences: true, go_backwards: true);
        model.add(keras.layers.Bidirectional(forwardLayer, merge_mode: "sum", backward_layer: backwardlayer));

Alternatives

No response

Wanglongzhi2001 commented

Yes, it has been support. You can refer to

TensorFlow.NET/test/TensorFlowNET.Keras.UnitTest/Layers/Rnn.Test.cs

Lines 158 to 165 in 8cab730

[TestMethod]
public void Bidirectional()
{
var bi = tf.keras.layers.Bidirectional(keras.layers.LSTM(10, return_sequences:true));
var inputs = tf.random.normal((32, 10, 8));
var outputs = bi.Apply(inputs);
Assert.AreEqual((32, 10, 20), outputs.shape);
}

But it may not have tested all parameters and situations like you set go_backwards to true in backward_layer here. So it may have some bugs here. And could you mind to provide more information, such as a minimal code for reproduction and more error information?

jaroslav-hook commented

after the model is defined, it model.summary() retuns zero parameters for the bidirectional layers
this gives zro parameters i summary public static void buildModel2()
{
//int embed_dim = 256;
int lstm_out = 196;
// var max_features = 4000;
var model = keras.Sequential();
//model.add(keras.layers.Embedding(max_features, embed_dim, input_length : 35));
model.add(keras.layers.Dropout(0.4f)); // SpatialDropout1D
model.add(keras.layers.Bidirectional(keras.layers.LSTM(lstm_out, dropout: 0.05f, recurrent_dropout: 0.2f))); // Add f
model.add(keras.layers.Dense(2, activation : keras.activations.Softmax)); // replace 'Softmax' with keras.activations.Softmax
model.compile(loss: "MeanSquaredError", optimizer: "adam", metrics: new string[] { "accuracy" });
model.summary();
}

jaroslav-hook commented 
    This  gives no parameters   in model.summary()

public static void buildModel2()
{
//int embed_dim = 256;
int lstm_out = 196;
// var max_features = 4000;
var model = keras.Sequential();
//model.add(keras.layers.Embedding(max_features, embed_dim, input_length : 35));
model.add(keras.layers.Dropout(0.4f)); // SpatialDropout1D
model.add(keras.layers.Bidirectional(keras.layers.LSTM(lstm_out, dropout: 0.05f, recurrent_dropout: 0.2f))); // Add f
model.add(keras.layers.Dense(2, activation : keras.activations.Softmax)); // replace 'Softmax' with keras.activations.Softmax
model.compile(loss: "MeanSquaredError", optimizer: "adam", metrics: new string[] { "accuracy" });
model.summary();
}

MaSobkowiak commented

I encountered a similar problem, did you manage to find a solution?

jaroslav-hook commented
MaSobkowiak commented

If you dont mind me asking, do you find any better alternative, or you stick with Python version?

Maybe have you tried using python inside .NET with Python.NET?

jaroslav-hook commented