Is it possible to guide inpainting using controlnet?
i-amgeek opened this issue · 16 comments
Model/Pipeline/Scheduler description
Hi, I am trying to inpaint a person in specific pose. Is it possible to use controlnet with this repo?
Open source status
- The model implementation is available.
- The model weights are available (Only relevant if addition is not a scheduler).
Provide useful links for the implementation
No response
Hi, @i-amgeek ! Thank you for your focus. I think it is theoretically sound to combine ControlNet and BrushNet together. But I haven't tried that personally. I may give it a try if I have the time. Moreover, we are welcome for any PR or discussion. Please tell us if you have tried that!
StableDiffusionBrushNetPipeline can load controlnet? or does it need to be modified?
I have retained the ControlNet functionality in this repo, but I am not sure whether it would work normally since I haven't tried that. If it works, please let me know! I will find time to try that maybe after a few days.
I have tried combining with controlnet，it works! need modified StableDiffusionBrushNetPipeline.
Hi, @jiechu520 ! Is that possible for you to pull your modification of StableDiffusionBrushNetPipeline to this repo and give us some examples? Thank you so much for your response!
same!!! I want to use controlnet with brushnet. can you upload the code? I think it will be really nice quality with controlnet. because unother paper replace anything use controlnet
I have tried combining with controlnet，it works! need modified StableDiffusionBrushNetPipeline.
Hi, can you show the input and output?
I tried combining it with controlnet and got unsatisfactory output.
@YoucanBaby can you share your code? I will test and fix the issue
The node in comfyui has supported this. Issue closed.