tensorflow/tensorrt

What is the statefulpartitionedcall_args with dtype DT_RESOURCE after convert to Tensorrt ?

dathudeptrai opened this issue · 7 comments

dathudeptrai commented

I'm trying to convert my model to tensorrt. My model have 2 inputs but after convert to tensorrt, my pb model have 2 more inputs with dtype is DT_RESOURCE so i don't know how to infer my model, pls help me check, i include my notebook link here: https://colab.research.google.com/drive/1KG_ttmlA45t5O1p7geTAjfR1htCrwfHU

sanjoy commented

CC @bixia1 @tfeher

JFYI @dathudeptrai it will be useful if you narrowed your example to something small that still reproduces the problem.

tfeher commented

That sounds like a bug in setting up the input signatures for the converted model. The node names can change during conversion (statefulpartitionedcall prefix added), but the signatures should not. I will have a look.

tfeher commented

Hi, it is a problem while creating the original SavedModel or while freezing that model. Try TF 2.1, that works for me.

Note that TF 2.1 gives you a warning about name mangling:

WARNING:tensorflow:AutoGraph could not transform <bound method TFTacoEncoder.call of <__main__.TFTacoEncoder object at 0x7fd9bd2218d0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Mangled names are not yet supported by AutoGraph

Do not use douple leading underscore in your method names (unless you really need that). This introduces name mangling which is not handled by AutoGraph. To fix that, just replace __get_non_pad_mask with _get_non_pad_mask.

  • 👍3
dathudeptrai commented

@tfeher thanks for ur reply, so u use TF 2.1 with tensorrt 6.0 ?

tfeher commented

I use TF 2.1 that I compiled from source with TensorRT 7.0. In practice, if you do

!pip install tensorflow-gpu==2.1.0

in the colab notebook, that package expects TRT 6.0.1, which you can install by:

%%bash
wget https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/libnvinfer6_6.0.1-1+cuda10.1_amd64.deb
dpkg -i libnvinfer6_6.0.1-1+cuda10.1_amd64.deb
dathudeptrai commented

Thanks, now i can covert it to tensorrt.