After using the tf-trt, there is nothing happen.
chiehpower opened this issue · 0 comments
chiehpower commented
Hi,
I was following this instruction.
https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html#usage-example
This is my code.
from tensorflow.python.compiler.tensorrt import trt_convert as trt
input_saved_model_dir = '/home/chieh/Downloads/TensorRT-7.0.0.11/samples/python/mrcnn/mask_rcnn_inception_v2_coco_2018_01_28/saved_model/'
output_saved_model_dir = '/home/chieh/Downloads/TensorRT-7.0.0.11/samples/python/mrcnn/mask_rcnn_inception_v2_coco_2018_01_28/trt_output/'
converter = trt.TrtGraphConverter(input_saved_model_dir=input_saved_model_dir)
converter.convert()
converter.save(output_saved_model_dir)
Then I got this output messages.
mrcnn ›› python3 trt_trying.py
2020-01-13 11:01:01.610801: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-01-13 11:01:01.615636: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcuda.so.1
2020-01-13 11:01:01.696094: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-01-13 11:01:01.696510: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x38c2f50 executing computations on platform CUDA. Devices:
2020-01-13 11:01:01.696528: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): GeForce GTX 1060 6GB, Compute Capability 6.1
2020-01-13 11:01:01.721289: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3400245000 Hz
2020-01-13 11:01:01.721858: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x3cd15f0 executing computations on platform Host. Devices:
2020-01-13 11:01:01.721875: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2020-01-13 11:01:01.722069: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-01-13 11:01:01.722366: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties:
name: GeForce GTX 1060 6GB major: 6 minor: 1 memoryClockRate(GHz): 1.759
pciBusID: 0000:01:00.0
2020-01-13 11:01:01.722550: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2020-01-13 11:01:01.722648: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda-10.2/lib64:/usr/local/cuda-10.2/lib64::/usr/local/cuda-10.2/lib64:/home/chieh/Downloads/TensorRT-7.0.0.11/lib:/usr/local/cuda-10.2/lib64:/home/chieh/Downloads/TensorRT-7.0.0.11/lib
2020-01-13 11:01:01.829945: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0
2020-01-13 11:01:01.861200: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0
2020-01-13 11:01:02.064355: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0
2020-01-13 11:01:02.102457: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0
2020-01-13 11:01:02.105418: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2020-01-13 11:01:02.105438: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1663] Cannot dlopen some GPU libraries. Skipping registering GPU devices...
2020-01-13 11:01:02.105454: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-01-13 11:01:02.105471: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187] 0
2020-01-13 11:01:02.105478: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0: N
WARNING:tensorflow:From /home/chieh/.local/lib/python3.6/site-packages/tensorflow/python/compiler/tensorrt/trt_convert.py:245: load (from tensorflow.python.saved_model.loader_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.loader.load or tf.compat.v1.saved_model.load. There will be a new function for importing SavedModels in Tensorflow 2.0.
WARNING:tensorflow:From /home/chieh/.local/lib/python3.6/site-packages/tensorflow/python/compiler/tensorrt/trt_convert.py:268: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.convert_variables_to_constants`
WARNING:tensorflow:From /home/chieh/.local/lib/python3.6/site-packages/tensorflow/python/framework/graph_util_impl.py:270: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.extract_sub_graph`
2020-01-13 11:01:04.869843: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-01-13 11:01:04.870227: I tensorflow/core/grappler/devices.cc:55] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 1
2020-01-13 11:01:04.870342: I tensorflow/core/grappler/clusters/single_machine.cc:359] Starting new session
2020-01-13 11:01:04.870783: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-01-13 11:01:04.871105: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties:
name: GeForce GTX 1060 6GB major: 6 minor: 1 memoryClockRate(GHz): 1.759
pciBusID: 0000:01:00.0
2020-01-13 11:01:04.871163: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2020-01-13 11:01:04.871285: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda-10.2/lib64:/usr/local/cuda-10.2/lib64::/usr/local/cuda-10.2/lib64:/home/chieh/Downloads/TensorRT-7.0.0.11/lib:/usr/local/cuda-10.2/lib64:/home/chieh/Downloads/TensorRT-7.0.0.11/lib
2020-01-13 11:01:04.871319: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0
2020-01-13 11:01:04.871341: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0
2020-01-13 11:01:04.871361: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0
2020-01-13 11:01:04.871370: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0
2020-01-13 11:01:04.871380: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2020-01-13 11:01:04.871395: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1663] Cannot dlopen some GPU libraries. Skipping registering GPU devices...
2020-01-13 11:01:04.871407: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-01-13 11:01:04.871413: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187] 0
2020-01-13 11:01:04.871429: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0: N
2020-01-13 11:01:06.764010: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:716] Optimization results for grappler item: tf_graph
2020-01-13 11:01:06.764051: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718] constant folding: Graph size after: 11371 nodes (-544), 19555 edges (-578), time = 1041.05603ms.
2020-01-13 11:01:06.764057: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718] layout: layout did nothing. time = 11.45ms.
2020-01-13 11:01:06.764061: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718] constant folding: Graph size after: 11371 nodes (0), 19555 edges (0), time = 387.413ms.
2020-01-13 11:01:08.186498: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-01-13 11:01:08.186924: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties:
name: GeForce GTX 1060 6GB major: 6 minor: 1 memoryClockRate(GHz): 1.759
pciBusID: 0000:01:00.0
2020-01-13 11:01:08.186990: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2020-01-13 11:01:08.187111: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda-10.2/lib64:/usr/local/cuda-10.2/lib64::/usr/local/cuda-10.2/lib64:/home/chieh/Downloads/TensorRT-7.0.0.11/lib:/usr/local/cuda-10.2/lib64:/home/chieh/Downloads/TensorRT-7.0.0.11/lib
2020-01-13 11:01:08.187147: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0
2020-01-13 11:01:08.187184: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0
2020-01-13 11:01:08.187205: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0
2020-01-13 11:01:08.187216: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0
2020-01-13 11:01:08.187249: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2020-01-13 11:01:08.187254: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1663] Cannot dlopen some GPU libraries. Skipping registering GPU devices...
2020-01-13 11:01:08.187281: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-01-13 11:01:08.187297: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187] 0
2020-01-13 11:01:08.187302: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0: N
However, there is no trt model in my output directory.
trt_output $ tree
.
├── saved_model.pb
└── variables
1 directory, 1 file
My model, mask_rcnn_inception_v2_coco_2018_01_28, was downloaded from here.
Is there any idea about my problem?
Thanks!
My env information
- TensorRT version: 7.0.0.11
- Cuda version: 10.2
- TensorFlow-gpu: 1.14.0
- Cudnn version: 7.6.5
- GPU: GTX1060
- Ubuntu: 18.04
- Python 3.6.9