CUDA error: an illegal memory access was encountered

[wkpark]

RuntimeError: CUDA error: an illegal memory access was encountered occurred after uddetailer processing was done.

Processing 1 detection of model A for output generation 1.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 12/12 [00:09<00:00,  1.20it/s]
Exception in thread MemMon:███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 40/40 [00:36<00:00,  1.18it/s]
Traceback (most recent call last):
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "F:\webui\webui\stable-diffusion-webui\modules\memmon.py", line 53, in run
    free, total = self.cuda_mem_get_info()
*** Error running postprocess_image: F:\webui\webui\stable-diffusion-webui\extensions\redetailer\scripts\ddetailer.py
  File "F:\webui\webui\stable-diffusion-webui\modules\memmon.py", line 34, in cuda_mem_get_info
    Traceback (most recent call last):
      File "F:\webui\webui\stable-diffusion-webui\modules\scripts.py", line 679, in postprocess_image
        script.postprocess_image(p, pp, *script_args)
      File "F:\webui\webui\stable-diffusion-webui\extensions\redetailer\scripts\ddetailer.py", line 2718, in postprocess_image
        self._postprocess_image(p, pp, use_prompt_edit, use_prompt_edit_2,
      File "F:\webui\webui\stable-diffusion-webui\extensions\redetailer\scripts\ddetailer.py", line 2556, in _postprocess_image
        processed = processing.process_images(p)
      File "F:\webui\webui\stable-diffusion-webui\modules\processing.py", line 734, in process_images
        res = process_images_inner(p)
      File "F:\webui\webui\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 42, in processing_process_images_hijack
        return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
      File "F:\webui\webui\stable-diffusion-webui\modules\processing.py", line 877, in process_images_inner
        x_samples_ddim = decode_latent_batch(p.sd_model, samples_ddim, target_device=devices.cpu, check_for_nans=True)
      File "F:\webui\webui\stable-diffusion-webui\modules\processing.py", line 600, in decode_latent_batch
        devices.test_for_nans(sample, "vae")
      File "F:\webui\webui\stable-diffusion-webui\modules\devices.py", line 118, in test_for_nans
        if not torch.all(torch.isnan(x)).item():
    RuntimeError: CUDA error: an illegal memory access was encountered
    CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
    For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
    Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

    return torch.cuda.mem_get_info(index)
  File "F:\webui\webui\stable-diffusion-webui\venv\lib\site-packages\torch\cuda\memory.py", line 618, in mem_get_info

---
    return torch.cuda.cudart().cudaMemGetInfo(device)
RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

[wkpark]

solved by commit 2e520718abdc64