lllyasviel/ControlNet

RuntimeError: Error(s) in loading state_dict for ControlLDM: Unexpected key(s) in state_dict: "cond_stage_model.transformer.text_model.embeddings.position_ids".

LukeLIN-web opened this issue · 4 comments

LukeLIN-web commented

Loaded state_dict from [./models/control_sd15_canny.pth]
Traceback (most recent call last):
File "/home/juyi/ControlNet/gradio_canny2image.py", line 21, in
model.load_state_dict(load_state_dict('./models/control_sd15_canny.pth', location='cuda'))
File "/home/juyi/miniconda3/envs/viditq/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2152, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for ControlLDM:
Unexpected key(s) in state_dict: "cond_stage_model.transformer.text_model.embeddings.position_ids".

ButoneDream commented

me too

ButoneDream commented

Traceback (most recent call last):
File "gradio_pose2image.py", line 21, in
model.load_state_dict(load_state_dict('./models/control_sd15_openpose.pth', location='cuda'))
File "/data/zhoulei/anaconda3/envs/control/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1671, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for ControlLDM:
Unexpected key(s) in state_dict: "cond_stage_model.transformer.text_model.embeddings.position_ids".

yousaLING commented

Hi, maybe you should degrade the vision of transformers to 4.19.2.

  • 👍3
  • ❤️1
ChlaegerIO commented

I need the newest version of transformers library for some other task. Thus degrading to 4.19.2 is not an option for me. I found that setting the strict attribute to false also does the job.

model.load_state_dict(load_state_dict(f'.<path to model>', location='cuda'), strict=False))

I however have not looked at what exactly changed in one of the four models used from transformers in ldm\modules\encoders\modules.py

from transformers import T5Tokenizer, T5EncoderModel, CLIPTokenizer, CLIPTextModel
  • 👍2