Installation of PyTorch in `devel` container causes `nvinferserver` to fail to load
aihsani opened this issue · 0 comments
aihsani commented
Any attempt to install torch
will cause nvinferserver
to fail to load.
Repro
- Run
start_devel.sh
- In the container run
main.py
insample/monai-stream-app
Expected
NVInferServer loads correctly
Actual
ERROR: infer_trtis_server.cpp:981 Triton: failed to create repo server, triton_err_str:Not found, err_msg:unable to load custom library: /opt/tritonserver/backends/pytorch/libtorchvision.so: undefined symbol: _ZNK5torch8autograd4Node4nameB5cxx11Ev
ERROR: infer_trtis_server.cpp:827 failed to initialize trtserver on repo dir: root: "/app/models"
strict_model_config: true
0:00:00.753716734 5706 0x54e4270 ERROR nvinferserver gstnvinferserver.cpp:362:gst_nvinfer_server_logger:<2eec0da80738494591863b5536de16a4-inference> nvinferserver[UID 1]: Error in createNNBackend() <infer_trtis_context.cpp:224> [UID = 1]: model:monai_unet_pytorch get triton server instance failed. repo:root: "/app/models"
strict_model_config: true
0:00:00.753739512 5706 0x54e4270 ERROR nvinferserver gstnvinferserver.cpp:362:gst_nvinfer_server_logger:<2eec0da80738494591863b5536de16a4-inference> nvinferserver[UID 1]: Error in initialize() <infer_base_context.cpp:81> [UID = 1]: create nn-backend failed, check config file settings, nvinfer error:NVDSINFER_TRTIS_ERROR
Workaround
pip uninstall torch
allows NVInferServer to operate correctly