TF 0.9: "This module is deprecated. Use tf.nn.rnn_* instead."

[hunkim]

I think we need to update lstm:

 File "07_lstm.py", line 3, in <module>
    from tensorflow.models.rnn import rnn, rnn_cell
  File "/home/travis/virtualenv/python2.7.10/lib/python2.7/site-packages/tensorflow/models/rnn/rnn.py", line 21, in <module>
    raise ImportError("This module is deprecated.  Use tf.nn.rnn_* instead.")
ImportError: This module is deprecated.  Use tf.nn.rnn_* instead.

[hunkim]

rnn_cell->tf.nn.rnn_cell
rnn.rnn->tf.nn.rnn

But got this warning:

WARNING:tensorflow:<tensorflow.python.ops.rnn_cell.BasicLSTMCell object at 0x101e360b8>: 
Using a concatenated state is slower and will soon be deprecated.  Use state_is_tuple=True.

Need to inspect more. Please let me know if you have some ideas.

[nlintz]

I'm not positive but I think that I have an idea of whats going on. For the initial state of the network (lets say we're using a gru), tensorflow can pass in one matrix of shape (batch_size x hidden_dim) which represents the z_state and another matrix of shape (batch_size x hidden_dim) representing the r_state. Instead, it could pass in a single matrix of size (batch_size x 2 * hidden dim), perform the dot product and slice out the r and z components of the single hidden dim. Usually the latter is faster but I guess that tensorflow must have some new optimization making it faster to have two separate state matrices instead of a single one.

Do you get an error when you set state_is_tuple=True? If not, this might be the correct way to do rnns in TF from now on.

[hunkim]

Thanks! I think I'll have time to look into this problem next week.

[Laserbear]

was a solution found?

[hunkim]

I think #46 fixed it. Please let us know if it does not work. Thanks!