guoqincode/Open-AnimateAnyone

mat1 and mat2 shapes cannot be multiplied (4x768 and 1280x320)

xiaohutongxue-sunny opened this issue · 2 comments

Traceback (most recent call last):
File "my_track.py", line 661, in
main(name=name, launcher=args.launcher, use_wandb=args.wandb, **config)
File "my_track.py", line 519, in main
referencenet(latents_ref_img, ref_timesteps, encoder_hidden_states)
File "/usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/home/open-AnimateAnyone/models/ReferenceNet.py", line 1009, in forward
sample, res_samples = downsample_block(
File "/usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/usr/local/lib/python3.8/site-packages/diffusers/models/unet_2d_blocks.py", line 1095, in forward
hidden_states = attn(
File "/usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/usr/local/lib/python3.8/site-packages/diffusers/models/transformer_2d.py", line 318, in forward
hidden_states = block(
File "/usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/home/open-AnimateAnyone/models/ReferenceNet_attention.py", line 202, in hacked_basic_transformer_inner_forward
attn_output = self.attn2(
File "/usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/usr/local/lib/python3.8/site-packages/diffusers/models/attention_processor.py", line 417, in forward
return self.processor(
File "/usr/local/lib/python3.8/site-packages/diffusers/models/attention_processor.py", line 587, in call
key = attn.to_k(encoder_hidden_states, scale=scale)
File "/usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/usr/local/lib/python3.8/site-packages/diffusers/models/lora.py", line 224, in forward
out = super().forward(hidden_states)
File "/usr/local/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 115, in forward
return F.linear(input, self.weight, self.bias)
RuntimeError: mat1 and mat2 shapes cannot be multiplied (4x768 and 1280x320)

would you help me. i don't kown why. Thank you.

Hello, have you changed the code? I use the same code that is trainable.

Have u solved it? I encountered the same problem.