apple/ml-gmpi

Problem in Importing upfirdn2d_plugin.so

Closed this issue · 2 comments

OldDreamInWind commented

hi, when I try to run this command:

export PYTHONPATH=${GMPI_ROOT}:${GMPI_ROOT}/gmpi/models:$PYTHONPATH && \
python ${GMPI_ROOT}/gmpi/eval/vis/render_video.py \
--ckpt_path ${OUTPUT_DIR}/generator.pth \
--output_dir ${OUTPUT_DIR} \
--seeds ${SEED} \
--nplanes 96 \
--truncation_psi ${TRUNCATION_PSI} \
--exp_config ${OUTPUT_DIR}/config.pth \
--render_single_image 1

Then I meet this error, and my cuda version is 11.1

ImportError: /data/run01/scz5914/.cache/torch_extensions/upfirdn2d_plugin/upfirdn2d_plugin.so: 
undefined symbol: _ZNSt15__exception_ptr13exception_ptr9_M_addrefEv
warnings.warn('Failed to build CUDA kernels for upfirdn2d. Falling back to slow reference implementation. Details:\n\n' + traceback.format_exc())

Fortunately, this error doesn't block the render image generating.
But I really hope to get faster inference speed.
So, do you know how to solve this problem?

PS: full error info

/HOME/scz5914/run/huangan_work/ml-gmpi/gmpi/models/torch_utils/ops/upfirdn2d.py:34: UserWarning: Failed to build CUDA kernels for upfirdn2d. Falling back to slow reference implementation. Details:

Traceback (most recent call last):
  File "/HOME/scz5914/run/huangan_work/ml-gmpi/gmpi/models/torch_utils/ops/upfirdn2d.py", line 32, in _init
    _plugin = custom_ops.get_plugin('upfirdn2d_plugin', sources=sources, extra_cuda_cflags=['--use_fast_math'])
  File "/HOME/scz5914/run/huangan_work/ml-gmpi/gmpi/models/torch_utils/custom_ops.py", line 110, in get_plugin
    torch.utils.cpp_extension.load(name=module_name, verbose=verbose_build, sources=sources, **build_kwargs)
  File "/HOME/scz5914/.conda/envs/gmpi/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1092, in load
    keep_intermediates=keep_intermediates)
  File "/HOME/scz5914/.conda/envs/gmpi/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1318, in _jit_compile
    return _import_module_from_library(name, build_directory, is_python_module)
  File "/HOME/scz5914/.conda/envs/gmpi/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1701, in _import_module_from_library
    module = importlib.util.module_from_spec(spec)
  File "<frozen importlib._bootstrap>", line 583, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 1043, in create_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ImportError: /data/run01/scz5914/.cache/torch_extensions/upfirdn2d_plugin/upfirdn2d_plugin.so: undefined symbol: _ZNSt15__exception_ptr13exception_ptr9_M_addrefEv
Xiaoming-Zhao commented

Thanks a lot for your interest in this work.

May I know the OS version you are using? It seems like someone happened to encounter the same error as yours when using Ubuntu 21 (see this issue). If this is the case, maybe you need to install some GCC with versions different from the default one.

Hope this helps.

  • 👍1
OldDreamInWind commented

Thanks for your help. I see the issue, and find that maybe the gcc version and cuda version cause this error.
So, I update gcc to 11.2 and cuda to 11.6. Then I recreate my env and run the command, no error shows.

Thank you again.

  • 👍1