TF-TRT can not deal with dynamic_rnn, right?
MachineJeff opened this issue · 2 comments
Hi
I have tested TensoRT in my model so mant times, but always failed.
After debug the model, I suspect that TensorRT can not deal with dynamic_rnn, because in dynamic_rnn, tf.while_loop is called.
(Duplicate of NVIDIA/TensorRT#226)
Hi @MachineJeff,
Unfortunately the parsers do not currently support this. This is related to: NVIDIA/TensorRT#220 (comment)
At the moment, I think it is possible to implement directly through the TensorRT API, but you would need to save your model's weights (numpy arrays), create the model from scratch using the TensorRT API, and then load them: https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/python_api/infer/Graph/Layers.html#rnn-layers
Solved.
disscussion is in NVIDIA/TensorRT#226.