- Authors: Tao Yu, Runseng Feng, Ruoyu Feng, Jinming Liu, Xin Jin, Wenjun Zeng and Zhibo Chen.
- Institutes: University of Science and Technology of China; Eastern Institute for Advanced Study.
- Paper: arXiv
TL; DR: Users can select any object in an image by clicking on it. With powerful vision models, e.g., SAM, LaMa and Stable Diffusion (SD), Inpaint Anything is able to remove the object smoothly (i.e., Remove Anything). Further, prompted by user input text, Inpaint Anything can fill the object with any desired content (i.e., Fill Anything) or replace the background of it arbitrarily (i.e., Replace Anything).
- Remove Anything
- Fill Anything
- Replace Anything
- Any aspect ratio supported
- 2K resolution supported
- Technical report on arXiv
- Demo Website (coming soon)
Click on an object in the image, and Inpainting Anything will remove it instantly!
- Click on an object;
- Segment Anything Model (SAM) segments the object out;
- Inpainting models (e.g., LaMa) fill the "hole".
Requires python>=3.8
python -m pip install torch torchvision torchaudio
python -m pip install -e segment_anything
python -m pip install -r lama/requirements.txt Download the model checkpoints provided in segment_anything and lama (e.g. sam_vit_h_4b8939.pth and big-lama).
Specify an image and a point, and Inpaint-Anything will remove the object at the point.
python remove_anything.py \
--input_img ./example/remove-anything/dog.jpg \
--point_coords 200 450 \
--point_labels 1 \
--dilate_kernel_size 15 \
--output_dir ./results \
--sam_model_type "vit_h" \
--sam_ckpt sam_vit_h_4b8939.pth \
--lama_config ./lama/configs/prediction/default.yaml \
--lama_ckpt big-lama |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Text prompt: "a teddy bear on a bench"
Click on an object, type in what you want to fill, and Inpaint Anything will fill it!
- Click on an object;
- SAM segments the object out;
- Input a text prompt;
- Text-prompt-guided inpainting models (e.g., Stable Diffusion) fill the "hole" according to the text.
Requires python>=3.8
python -m pip install torch torchvision torchaudio
python -m pip install -e segment_anything
python -m pip install diffusers transformers accelerate scipy safetensorsDownload the model checkpoints provided in segment_anything (e.g. sam_vit_h_4b8939.pth).
Specify an image, a point and text prompt, and run:
python fill_anything.py \
--input_img ./example/fill-anything/sample1.png \
--point_coords 750 500 \
--point_labels 1 \
--text_prompt "a teddy bear on a bench" \
--dilate_kernel_size 50 \
--output_dir ./results \
--sam_model_type "vit_h" \
--sam_ckpt sam_vit_h_4b8939.pth |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Text prompt: "a man in office"
Click on an object, type in what background you want to replace, and Inpaint Anything will replace it!
- Click on an object;
- SAM segments the object out;
- Input a text prompt;
- Text-prompt-guided inpainting models (e.g., Stable Diffusion) replace the background according to the text.
Requires python>=3.8
python -m pip install torch torchvision torchaudio
python -m pip install -e segment_anything
python -m pip install diffusers transformers accelerate scipy safetensorsDownload the model checkpoints provided in segment_anything (e.g. sam_vit_h_4b8939.pth).
Specify an image, a point and text prompt, and run:
python fill_anything.py \
--input_img ./example/replace-anything/dog.png \
--point_coords 750 500 \
--point_labels 1 \
--text_prompt "sit on the swing" \
--output_dir ./results \
--sam_model_type "vit_h" \
--sam_ckpt sam_vit_h_4b8939.pth |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
If you find this work useful for your research, please cite us:
@misc{yu2023inpaint,
title={Inpaint Anything: Segment Anything Meets Image Inpainting},
author={Tao Yu and Runseng Feng and Ruoyu Feng and Jinming Liu and Xin Jin and Wenjun Zeng and Zhibo Chen},
year={2023},
eprint={2304.06790},
archivePrefix={arXiv},
primaryClass={cs.CV}
}