How can we load lora model(just a single file as .safetensors)?
Ethan-Chen-plus opened this issue · 11 comments
How can we load lora model(just a single file as .safetensors)?
If the model is available on HuggingFace 🤗, you could change the model_id. Otherwise, this would probably require more work.
I could find:
Can I use relative paths? like "./models/mylora.safetensors"
And another question is, I used webui before,but I prefer jupyter notebook or cli because I can see how these codes work. I have learnt that lora have a base model. Do we need load both the base model and the lora weights at the same time? And how to use
Can I use relative paths? like "./models/mylora.safetensors"
This would probably require more work, e.g. to copy what happens behind the scene in StableDiffusionPipeline.
Do we need load both the base model and the lora weights at the same time? And how to use
I believe you can use:
model_id = "prompthero/openjourney-lora"I will see if the notebook works with this model_id. Edit: This does not work. 😞
HTTPError: 404 Client Error: Not Found for url:
https://huggingface.co/prompthero/openjourney-lora/resolve/main/model_index.json
@woctezuma I see this code in huggingface:
import torch
from diffusers import StableDiffusionPipeline, DPMSolverMultistepScheduler
from huggingface_hub import model_info
# LoRA weights ~3 MB
model_path = "pcuenq/pokemon-lora"
info = model_info(model_path)
model_base = info.cardData["base_model"]
pipe = StableDiffusionPipeline.from_pretrained(model_base, torch_dtype=torch.float16)
pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config)
pipe.unet.load_attn_procs(model_path)
pipe.to("cuda")
image = pipe("Green pokemon with menacing face", num_inference_steps=25).images[0]
image.save("green_pokemon.png")
https://huggingface.co/pcuenq/pokemon-lora/blob/main/README.md
I am wondering whether it suits safetensors.
Interesting. It looks like this would answer your question.
Do we need load both the base model and the lora weights at the same time? And how to use.
One would have to first load the base model:
info = model_info(model_path)
model_base = info.cardData["base_model"]
pipe = StableDiffusionPipeline.from_pretrained(model_base, torch_dtype=torch.float16)then load the weights with:
# LoRA weights ~3 MB
model_path = "pcuenq/pokemon-lora"
pipe.unet.load_attn_procs(model_path)
maybe it can't load ckpt.
and it cann't load safetensors.
huggingface/diffusers#3064 (comment)
this will work.😁
huggingface/diffusers#3064 (comment) this will work.😁
Wow, that required quite some work! 😮 I will try it when I have the opportunity! Thanks for sharing the link with me!
@woctezuma Thanks a lot! My problem has been solved.
Thank you! You did all the research! 🙏 Good job solving the issue! 🥳 🎉