Error when using fuse_lora

[congdc00]

Hello everyone,

This repo is awesome, and I really need it right now. I ran your code as follows:

self.pipe = CogVideoXImageToVideoPipeline.from_pretrained("THUDM/CogVideoX-5b-I2V", torch_dtype=torch.bfloat16)
lora_path = "/app/models"
lora_rank = 256
self.pipe.load_lora_weights(lora_path, weight_name="orbit_left_lora_weights.safetensors", adapter_name="orbit_left")
self.pipe.fuse_lora(adapter_names=["orbit_left"], lora_scale=1 / lora_rank)
self.pipe.to("cuda")

And I received this error:

File "/app/src/video_generator/__init__.py", line 7, in <module>
    model_genvid = CogVideo(configs)
                   ^^^^^^^^^^^^^^^^^
  File "/app/src/video_generator/cog/__init__.py", line 82, in __init__
    self.pipe.fuse_lora(adapter_names=["orbit_left"], lora_scale=1 / lora_rank)
  File "/usr/local/lib/python3.11/dist-packages/diffusers/loaders/lora_pipeline.py", line 2888, in fuse_lora
    super().fuse_lora(
  File "/usr/local/lib/python3.11/dist-packages/diffusers/loaders/lora_base.py", line 445, in fuse_lora
    raise ValueError(f"{fuse_component} is not found in {self._lora_loadable_modules=}.")
ValueError: text_encoder is not found in self._lora_loadable_modules=['transformer'].

The libraries I am using:

Package	Version
absl-py	2.1.0
accelerate	1.1.1
aiodns	3.2.0
aiohappyeyeballs	2.4.3
aiohttp	3.10.10
aiohttp-retry	2.9.0
aiosignal	1.3.1
annotated-types	0.7.0
anyio	4.6.0
argon2-cffi	23.1.0
argon2-cffi-bindings	21.2.0
arrow	1.3.0
asttokens	2.4.1
async-lru	2.0.4
attrs	24.2.0
babel	2.16.0
backoff	2.2.1
bcrypt	4.2.0
beautifulsoup4	4.12.3
bleach	6.1.0
blinker	1.4
boto3	1.35.54
botocore	1.35.54
Brotli	1.1.0
certifi	2024.8.30
cffi	1.17.1
chardet	5.2.0
charset-normalizer	3.3.2
click	8.1.7
colorama	0.4.6
comm	0.2.2
cryptography	3.4.8
DataProperty	1.0.1
datasets	3.1.0
dbus-python	1.2.18
debugpy	1.8.5
decorator	4.4.2
defusedxml	0.7.1
diffuser	0.0.1
diffusers	0.31.0
dill	0.3.8
distro	1.7.0
dnspython	2.7.0
email-validator	2.2.0
entrypoints	0.4
evaluate	0.4.3
executing	2.1.0
fastapi	0.115.4
fastapi-cli	0.0.5
fastjsonschema	2.20.0
filelock	3.13.1
fqdn	1.5.1
frozenlist	1.5.0
fsspec	2024.2.0
gdown	5.2.0
greenlet	1.1.2
h11	0.14.0
httpcore	1.0.5
httplib2	0.20.2
httptools	0.6.4
httpx	0.27.2
huggingface-hub	0.26.2
idna	3.10
imageio	2.36.0
imageio-ffmpeg	0.5.1
importlib-metadata	4.6.4
inquirerpy	0.3.4
ipykernel	6.29.5
ipython	8.27.0
ipython-genutils	0.2.0
ipywidgets	8.1.5
isoduration	20.11.0
itsdangerous	2.2.0
jedi	0.19.1
jeepney	0.7.1
Jinja2	3.1.3
jmespath	1.0.1
joblib	1.4.2
json5	0.9.25
jsonlines	4.0.0
jsonpointer	3.0.0
jsonschema	4.23.0
jsonschema-specifications	2023.12.1
jupyter-archive	3.4.0
jupyter_client	7.4.9
jupyter_contrib_core	0.4.2
jupyter_contrib_nbextensions	0.7.0
jupyter_core	5.7.2
jupyter-events	0.10.0
jupyter-highlight-selected-word	0.2.0
jupyter-lsp	2.2.5
jupyter_nbextensions_configurator	0.6.4
jupyter_server	2.14.2
jupyter_server_terminals	0.5.3
jupyterlab	4.2.5
jupyterlab_pygments	0.3.0
jupyterlab_server	2.27.3
jupyterlab_widgets	3.0.13
keyring	23.5.0
launchpadlib	1.10.16
lazr.restfulclient	0.14.4
lazr.uri	1.0.6
lm_eval	0.4.5
loguru	0.7.2
lxml	5.3.0
markdown-it-py	3.0.0
MarkupSafe	2.1.5
matplotlib-inline	0.1.7
mbstrdecoder	1.1.3
mdurl	0.1.2
mistune	3.0.2
more-itertools	8.10.0
moviepy	1.0.3
mpmath	1.3.0
msgpack	1.0.3
multidict	6.1.0
multiprocess	0.70.16
nbclassic

Please let me know how to resolve this issue. Thank all!

[chenshuo20]

Hi, you can take a look this issue.

[congdc00]

This worked, thank you <3