UniPortrait: A Unified Framework for Identity-Preserving Single- and Multi-Human Image Personalization

UniPortrait is an innovative human image personalization framework. It customizes single- and multi-ID images in a unified manner, providing high-fidelity identity preservation, extensive facial editability, free-form text description, and no requirement for a predetermined layout.

Release

[2024/10/18] 🔥 We release the inference code and demo, which has simply integrated ControlNet , IP-Adapter, and StyleAligned. The weight for this version is consistent with the huggingface space and experiments in the paper. We are now working on generalizing our method to more advanced diffusion models and more general custom concepts. Please stay tuned!
[2024/08/12] 🔥 We release the technical report , project page, and HuggingFace demo 🤗!

Quickstart

# Clone repository
git clone https://github.com/junjiehe96/UniPortrait.git

# install requirements
cd UniPortrait
pip install -r requirements.txt

# download the models
git lfs install
git clone https://huggingface.co/Junjie96/UniPortrait models
# download ip-adapter models 
# Note: recommend downloading manually. We do not require all IP adapter models.
git clone https://huggingface.co/h94/IP-Adapter models/IP-Adapter

# then you can use the gradio app
python gradio_app.py

Applications

Acknowledgements

This code is built on some excellent repos, including diffusers, IP-Adapter and StyleAligned. Highly appreciate their great work!

Cite

If you find UniPortrait useful for your research and applications, please cite us using this BibTeX:

@article{he2024uniportrait,
    title={UniPortrait: A Unified Framework for Identity-Preserving Single-and Multi-Human Image Personalization},
    author={He, Junjie and Geng, Yifeng and Bo, Liefeng},
    journal={arXiv preprint arXiv:2408.05939},
    year={2024}
}

For any question, please feel free to open an issue or contact us via hejunjie1103@gmail.com.