/image-anything

A demo of image models

Primary LanguagePythonOtherNOASSERTION

Image Anything

Open In Colab
A gradio demo of image models. Refer to colab notebook for setup.

The project incorporates image generation and editing models in concert to perform various tasks.

Currently Supported Tasks:

  1. Image captioning
  2. Generate auto SAM mask
  3. Objects detection, segmentation, annotation
  4. Remove / replace background
  5. Inpainting
  6. Upscale image 4x
  7. Text to image
  8. Drawing to image
  9. Image to image

Interaction modes:

  • Selecting points on the image
  • Text prompts
  • Auto mode
  • Drawing
  • Upload image mask -> TBD
  • Audio -> TBD

Models used:

  • Segment Anything (SAM)
  • Grounding DINO
  • Matte Anything (ViTMatte - Hust Labs)
  • Stable Diffusion 2 (Hugging Face diffusers)
  • Stable Diffusion Controlnet
  • BLIP
  • Mobile SAM
  • Matte Anything Model (MAM - SHI Labs) -> TBD

TBD:

  • Options to choose from checkpoints e.g. Stable Diffusion versions
  • Options to further control SD generation
  • More tasks e.g. image editing with more models

App Snaps

Auto SAM Mask: app screen

Upscale Task and Text to Image Task: txt2img upscale

Annotations: anns

Inpainting (tea pot -> puppy || green apple -> orange || cat -> rabbit):
inpaint

Remove/Replace Background (SD generated backgrounds):
bgr
Remove Background for Transparent objects: bg transp

Drawing to Image:
draw to img

Image to Image A. (prompt for terrace swimming pool): img to img 1

Image to Image B. (prompt for 1: pool table with balls, 2:fantasy landscape on artstation): img to img 2

Advanced Settings to tune the results:
settings

Acknowledgements

This app is built with the help of following models and libraries. Please visit their pages to know more about them.