Error occurred when executing DynamiCrafterI2V:

Question

Error occurred when executing DynamiCrafterI2V:

XTRMsavage opened this issue 7 months ago · 3 comments

Hi, I am getting this error, what's the cause ? is it XFORMERS?

No operator found for memory_efficient_attention_forward with inputs:
query : shape=(80, 7168, 1, 64) (torch.float16)
key : shape=(80, 7168, 1, 64) (torch.float16)
value : shape=(80, 7168, 1, 64) (torch.float16)
attn_bias :
p : 0.0
decoderF is not supported because:
xFormers wasn't build with CUDA support
attn_bias type is
operator wasn't built - see python -m xformers.info for more info
flshattF@0.0.0 is not supported because:
xFormers wasn't build with CUDA support
operator wasn't built - see python -m xformers.info for more info
cutlassF is not supported because:
xFormers wasn't build with CUDA support
operator wasn't built - see python -m xformers.info for more info
smallkF is not supported because:
max(query.shape[-1] != value.shape[-1]) > 32
xFormers wasn't build with CUDA support
dtype=torch.float16 (supported: {torch.float32})
operator wasn't built - see python -m xformers.info for more info
unsupported embed per head: 64

File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-0246\utils.py", line 381, in new_func
res_value = old_func(*final_args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\nodes.py", line 454, in process
samples, _ = ddim_sampler.sample(S=steps,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\models\samplers\ddim.py", line 119, in sample
samples, intermediates = self.ddim_sampling(conditioning, size,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\models\samplers\ddim.py", line 192, in ddim_sampling
outs = self.p_sample_ddim(img, cond, ts, index=index, use_original_steps=ddim_use_original_steps,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\models\samplers\ddim.py", line 228, in p_sample_ddim
e_t_cond = self.model.apply_model(x, t, c, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\models\ddpm3d.py", line 574, in apply_model
x_recon = self.model(x_noisy, t, **cond, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\models\ddpm3d.py", line 736, in forward
out = self.diffusion_model(xc, t, context=cc, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\modules\networks\openaimodel3d.py", line 583, in forward
h = module(h, emb, context=context, batch_size=b, frame_window_size=frame_window_size, frame_window_stride=frame_window_stride)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\modules\networks\openaimodel3d.py", line 41, in forward
x = layer(x, context)
^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\modules\attention.py", line 311, in forward
x = block(x, context=context, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\modules\attention.py", line 246, in forward
return checkpoint(self._forward, input_tuple, self.parameters(), self.checkpoint)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\common.py", line 94, in checkpoint
return func(*inputs)
^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\modules\attention.py", line 250, in _forward
x = self.attn1(self.norm1(x), context=context if self.disable_self_attn else None, mask=mask) + x
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\modules\attention.py", line 182, in efficient_forward
out = xformers.ops.memory_efficient_attention(q, k, v, attn_bias=None, op=None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\xformers\ops\fmha_init.py", line 268, in memory_efficient_attention
return memory_efficient_attention(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\xformers\ops\fmha_init.py", line 387, in _memory_efficient_attention
return memory_efficient_attention_forward(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\xformers\ops\fmha_init.py", line 403, in _memory_efficient_attention_forward
op = _dispatch_fw(inp, False)
^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\xformers\ops\fmha\dispatch.py", line 125, in _dispatch_fw
return _run_priority_list(
^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\xformers\ops\fmha\dispatch.py", line 65, in _run_priority_list
raise NotImplementedError(msg)

Answer 1 · 2024-06-07T12:30:41.000Z

Yes, your xformers installation isn't compatible with your torch install. Possibly just reinstalling xformers could help, if not then re-installing xformers compatible torch version is required, for example latest stable 2.3.1 works with latest xformers.

This is all assuming you have nvidia GPU, xformers will not work otherwise and should simply be uninstalled.

Uninstall xformers:
python_embeded\python.exe -m pip uninstall xformers
Install latest version without dependencies:
python_embeded\python.exe -m pip xformers --no-deps

If that doesn't help:

Uninstall current torch:
python_embeded\python.exe -m pip uninstall torch torchvision torchaudio
Install latest torch:
python_embeded\python.exe -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

Answer 2 · 2024-06-11T02:17:08.000Z

This worked for me:

Uninstall both:
pip uninstall xformers
pip uninstall torch torchvision torchaudio

Reinstall both:
Used xformers install from facebookresearch/xformers :::
pip3 install -U xformers --index-url https://download.pytorch.org/whl/cu121
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

Since the recommended command from fb page was pip3 i just used it for both. I'm fairly sure pip would have worked for both...

Answer 3 · 2024-06-11T02:55:42.000Z

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

Thanks, I got the xformers installed but torchvision fails due to it's file size. Is there a way I can download it separately and put in the right location?