Project-MONAI/MONAIStream

Installation of PyTorch in `devel` container causes `nvinferserver` to fail to load

aihsani opened this issue · 0 comments

Any attempt to install torch will cause nvinferserver to fail to load.

Repro

  1. Run start_devel.sh
  2. In the container run main.py in sample/monai-stream-app

Expected
NVInferServer loads correctly

Actual

ERROR: infer_trtis_server.cpp:981 Triton: failed to create repo server, triton_err_str:Not found, err_msg:unable to load custom library: /opt/tritonserver/backends/pytorch/libtorchvision.so: undefined symbol: _ZNK5torch8autograd4Node4nameB5cxx11Ev
ERROR: infer_trtis_server.cpp:827 failed to initialize trtserver on repo dir: root: "/app/models"
strict_model_config: true

0:00:00.753716734  5706      0x54e4270 ERROR          nvinferserver gstnvinferserver.cpp:362:gst_nvinfer_server_logger:<2eec0da80738494591863b5536de16a4-inference> nvinferserver[UID 1]: Error in createNNBackend() <infer_trtis_context.cpp:224> [UID = 1]: model:monai_unet_pytorch get triton server instance failed. repo:root: "/app/models"
strict_model_config: true

0:00:00.753739512  5706      0x54e4270 ERROR          nvinferserver gstnvinferserver.cpp:362:gst_nvinfer_server_logger:<2eec0da80738494591863b5536de16a4-inference> nvinferserver[UID 1]: Error in initialize() <infer_base_context.cpp:81> [UID = 1]: create nn-backend failed, check config file settings, nvinfer error:NVDSINFER_TRTIS_ERROR

Workaround

pip uninstall torch

allows NVInferServer to operate correctly