TensorRT can not deal with dynamic_rnn in TensorFlow, right?
Closed this issue · 5 comments
Description
#tensorflow/tensorrt#154 (comment)
Hi
This issue is very short.
I have tested TensoRT in my model so mant times, but always failed.
After debug the model, I suspect that TensorRT can not deal with dynamic_rnn, because in dynamic_rnn, tf.while_loop is called.
Hi @MachineJeff,
Unfortunately the parsers do not currently support this. This is related to: #220 (comment)
At the moment, I think (but I'm not sure) it is possible to implement directly through the TensorRT API, but you would need to save your model's weights (numpy arrays), create the model from scratch using the TensorRT API, and then load them: https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/python_api/infer/Graph/Layers.html#rnn-layers
Thank you so much.
God knows how complicated to locate this bug.
You remember that? this issus #193 I reported not long ago was caused by dynamic_rnn .
Maybe you can make a memo for others's iuuses related to this strang bug.
Hi @MachineJeff,
TensorRT 7 should be released within the next few days with added RNN support and expanded ONNX op support. Check out https://developer.nvidia.com/tensorrt for more info, there's a TRT7 description at the bottom of the page.
Hopefully this will solve your issues! If not, please open a new issue, thanks!
Great !
I will try to test it.
Thank you very very much !