MaskOnClick is a powerful extension for Stable Diffusion Web UI, leveraging the potential of Meta's Segment Anything, an open source ONNX model. It provides an interactive tool for image segmentation, enabling users to swiftly generate masks within an image. Designed specifically for image editing and enhancing purposes, this extension allows the users to highlight and edit specific areas of an image.
This extension is extremely useful when combined with the inpainting technique provided by Stable Diffusion. For example, users can use it to replace the clothes of a person in an image or edit any specific object or area. Written in Python 3.10 and Torch 2.0, MaskOnClick offers a simple yet efficient solution for advanced image processing tasks.
Make sure you have Stable Diffusion Web UI installed on your system.
To install as extension on Stable Diffusion WebUi from Automatic1111:
- Clone the repository under the extensions stable directory.
git clone https://github.com/iskewedI/mask-on-click.git- Move into the directory and then into the segment-anything folder
cd mask-on-click
cd segment-anything- Activate SD WebUI environment
path-to-sd-webui\venv\Scripts\activate.bat- Clone Segment Anything repo here
git clone https://github.com/facebookresearch/segment-anything .- Install packages and install SAM model (read below)
pip install -e .- Fully reload SD UI if you have it already opened
To install MaskOnClick, please follow these steps:
- Clone the repository
git clone https://github.com/iskewedI/mask-on-click.git- Move into the directory
cd mask-on-click- Install the required packages
pip install -r requirements.txt- Install the extension to your Stable Diffusion Web UI project following its specific extension installation guidelines. You can also just clone the repo inside the Stable Diffusion WebUI extensions directory and reload the WebUI.
You need to download the default SAM model from the https://github.com/facebookresearch/segment-anything#getting-started SAM's installation process, and then put it into the pretrained mask-on-click directory as sam_model.pth
After installing the extension, you can access the MaskOnClick functionality directly from the Stable Diffusion Web UI interface. This extension is loaded as a script (you have a dropdown below all sections like text2img and img2img to select those). Simply choose an image for editing, load the script and there you can segment the image. The mask will be automatically added to the img2img inpaint process.
For Auto's A1111 SD WebUI, you need to set the --disable-safe-unpickle command argument to true (just add --disable-safe-unpickle to the webui-user.bat file COMMANDLINE_ARGS= line). This is because some weird behavior with SD custom torch .load method and the segment anything repo.
Common process:
- Python 3.10
- PyTorch 2.0
- Gradio
- Meta's Segment Anything
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
- Fork the Project
- Create your Feature Branch (git checkout -b feature/AmazingFeature)
- Commit your Changes (git commit -m 'Add some AmazingFeature')
- Push to the Branch (git push origin feature/AmazingFeature)
- Open a Pull Request
Distributed under the MIT License. See LICENSE for more information.
Please feel free to contact if you have any questions or need further information.