/Pose-Segment-Anything

Marrying Pose Estimation model with Segment Anything & Stable Diffusion, Automatically Joint Detection , Segment and edit any part of your body.

Primary LanguageJupyter Notebook

Pose-Segment-Anything

The core idea of this project is inspired by Grounded-Segment-Anything. We combine the pose estimation models so as to make use of the point prompt of Segment Anything. And then we can edit any part of the body with inpaintingstable diffusion. A simple but interesting workflow pipeline for combining strong expert models.

Demo Preview

HR-SAM:Automatic Labeling Components of the human body

HR-SAM-Stable-Diffusion Inpainting:Editing any Components of the human body

WebUI_DEMO with gradio

First choose the pth of SAM and HRNet, then select the Components you want and click Seg with HR prompt. Then write your text prompt and diffusion with previous mask to edit it to whatever you want.

How to start

1.Check the requirements.txt

pip install -r requirements.txt

2.Download pth of SAM and HRNet

Here is the link:SAM. HRNet.

3.Execute demo of HR-SAM of HR-SAM-Diffusion with your own image.

4.Execute appHRSAMDIFF.py Have fun with WebUI and get your own 'Human body modification program'

Contributors

Yuming Qiao, Fanyi Wang, Yanhao Zhang