/Stable-DiffusionXL-using-DreamBooth-and-LoRA

This project is an implementation of fine-tuning an SDXL model using DreamBooth and LoRA on custom data of interior rooms to generate designs for your home.

Primary LanguageJupyter NotebookMIT LicenseMIT

Stable Diffusion XL using DreamBooth and LoRA

This project is an implementation of fine-tuning a foundation SDXL model using DreamBooth and LoRA on custom data of interior room designs to generate:

  • Text-to-image
  • Image-to-image
  • Image-to-video

Pipeline:

Text-to-image generation using fine-tuned LoRA

Prompt: A photo of TOK home, an Indian living room basking in Republic Day morning sun, adorned with saffron, white, and green decor, and vibrant festive accents. You can try out this model for inference here.

Image-to-image generation using fine-tuned LoRA

Input Image Output Image
compressed (2) (1) Gemini_Pro_Vision_App (2)

Stable Video Diffusion

Image to Video generation.

Resources:

  1. Try it yourself on Hugging Face: https://huggingface.co/NSTiwari/SDXL_LoRA_model
  2. Demo video: https://youtu.be/X_p0E2vBRjY
  3. Blog: https://aashi-dutt3.medium.com/ai-architect-design-your-homes-with-stable-diffusion-b1c61644c61e