The code is heavily based on the Prompt-to-Prompt, Null-Text Inversion [codebase], the MasaCtrl [codebase], and the DDPM-Inversion [codebase].
Input images can be downloaded through this link. Most images are from the StyleDiffusion, Null-Text Inversion, Imagic, MasaCtrl, and SVDiff papers.
Negative-Prompt Inversion (NPI):
from negative_prompt_inversion import main
main(
image_path="./images/cat_mirror.png",
prompt_src="a cat sitting next to a mirror",
prompt_tar="a silver cat sculpture sitting next to a mirror",
output_dir="./outputs/npi_real",
suffix="silver",
guidance_scale=7.5,
cross_replace_steps=0.8,
self_replace_steps=0.6,
blend_word=((('cat',), ("cat",))),
eq_params={"words": ("silver", 'sculpture', ), "values": (2,2,)},
proximal=None,
)NPI reconstructed (left) | NPI edited (right):
To run ProxNPI:
from negative_prompt_inversion import main
main(
image_path="./images/cat_mirror.png",
prompt_src="a cat sitting next to a mirror",
prompt_tar="a silver cat sculpture sitting next to a mirror",
output_dir="./outputs/npi_real",
suffix="silver",
guidance_scale=7.5,
cross_replace_steps=0.8,
self_replace_steps=0.6,
blend_word=((('cat',), ("cat",))),
eq_params={"words": ("silver", 'sculpture', ), "values": (2,2,)},
proximal='l0',
quantile=0.7,
use_inversion_guidance=True,
recon_lr=1,
recon_t=400,
)Reconstructed (left) | Edited (right):
To run MasaCtrl:
from prox_masactrl import main
main(
out_dir="./outputs/masactrl_real/",
source_image_path="./images/cake2.png",
source_prompt="a round cake",
target_prompt="a square cake",
npi=False,
npi_interp=0,
)Input image (left) | Reconstructed (middle) | Edited (right):
To run ProxMasaCtrl:
from prox_masactrl import main
main(
out_dir="./outputs/masactrl_real/",
source_image_path="./images/cake2.png",
source_prompt="a round cake",
target_prompt="a square cake",
npi=True,
npi_interp=1,
prox='l0',
quantile=0.6,
)Input image (left) | Reconstructed (middle) | Edited (right):
Please see run_npi.py, run_masa.py and run_ddpm.py for examples.
@article{han2023improving,
title={Improving Negative-Prompt Inversion via Proximal Guidance},
author={Han, Ligong and Wen, Song and Chen, Qi and Zhang, Zhixing and Song, Kunpeng and Ren, Mengwei and Gao, Ruijiang and Chen, Yuxiao and Liu, Di and Zhangli, Qilong and others},
journal={arXiv preprint arXiv:2306.05414},
year={2023}
}@article{miyake2023negative,
title={Negative-prompt Inversion: Fast Image Inversion for Editing with Text-guided Diffusion Models},
author={Miyake, Daiki and Iohara, Akihiro and Saito, Yu and Tanaka, Toshiyuki},
journal={arXiv preprint arXiv:2305.16807},
year={2023}
}@inproceedings{mokady2023null,
title={Null-text inversion for editing real images using guided diffusion models},
author={Mokady, Ron and Hertz, Amir and Aberman, Kfir and Pritch, Yael and Cohen-Or, Daniel},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={6038--6047},
year={2023}
}@article{hertz2022prompt,
title={Prompt-to-prompt image editing with cross attention control},
author={Hertz, Amir and Mokady, Ron and Tenenbaum, Jay and Aberman, Kfir and Pritch, Yael and Cohen-Or, Daniel},
journal={arXiv preprint arXiv:2208.01626},
year={2022}
}@article{cao2023masactrl,
title={MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing},
author={Cao, Mingdeng and Wang, Xintao and Qi, Zhongang and Shan, Ying and Qie, Xiaohu and Zheng, Yinqiang},
journal={arXiv preprint arXiv:2304.08465},
year={2023}
}@article{song2020denoising,
title={Denoising diffusion implicit models},
author={Song, Jiaming and Meng, Chenlin and Ermon, Stefano},
journal={arXiv preprint arXiv:2010.02502},
year={2020}
}@article{HubermanSpiegelglas2023,
title = {An Edit Friendly DDPM Noise Space: Inversion and Manipulations},
author = {Huberman-Spiegelglas, Inbar and Kulikov, Vladimir and Michaeli, Tomer},
journal = {arXiv preprint arXiv:2304.06140},
year = {2023}
}