`Critical error when building pipeline`
strarsis opened this issue · 1 comments
strarsis commented
image-checker
{'path': './', 'batch_size': 50, 'device_id': 0, 'extensions': ['jpeg', 'jpg', 'png'], 'log_conf': 'logging_config.json', 'recursive': False, 'debug': False, 'use_cpu': False, 'prefetch': 1, 'device': 'mixed'}
Good images (±50): 0it [00:00,/usr/local/lib/python3.8/dist-packages/nvidia/dali/ops.py:649: DeprecationWarning: WARNING: `image_decoder` is now deprecated. Use `decoders.image` instead.
In DALI 1.0 all decoders were moved into a dedicated :mod:`~nvidia.dali.fn.decoders`
submodule and renamed to follow a common pattern. This is a placeholder operator with identical
functionality to allow for backward compatibility.
op_instances.append(_OperatorInstance(input_set, self, **kwargs))
Traceback (most recent call last):
File "/home/build/.local/bin/image-checker", line 8, in <module>
sys.exit(main())
File "/home/build/.local/lib/python3.8/site-packages/image_checker/cli.py", line 73, in main
checker_batch(ds, args)
File "/home/build/.local/lib/python3.8/site-packages/image_checker/checker.py", line 38, in checker_batch
checker = DaliChecker(args["batch_size"], args["prefetch"],args["device"],args["device_id"])
File "/home/build/.local/lib/python3.8/site-packages/image_checker/DaliChecker.py", line 27, in __init__
self.pipe.build()
File "/usr/local/lib/python3.8/dist-packages/nvidia/dali/pipeline.py", line 828, in build
self._pipe.Build(self._generate_build_args())
RuntimeError: Critical error when building pipeline:
Error when constructing operator: ImageDecoder encountered:
Error in thread 0: nvml error (3): The nvml requested operation is not available on target device
Current pipeline object is no longer valid.
Bad images: 0it [00:02, ?it/s]
Good images (±50): 0it [00:02, ?it/s]
This runs on WSL 2 with NVIDIA drivers and CUDA 11 (11.0.228
) support installed and available.
Or does Dali require explicit JPEG/image hardware decoder support on the GPU, which isn't present or passed-through on this system?
strarsis commented
Apparently this is an issue with the current NVML support on WSL 2. Disabling NVML for DALI by setting the environment variable DALI_DISABLE_NVML=1
fixes the issue.
DALI_DISABLE_NVML=1 image-checker
(Or use export DALI_DISABLE_NVML=1
or add DALI_DISABLE_NVML=1
to your e.g. .bashrc
).