/diffused-texture-addon

Generate Diffuse Textures on Meshes with Stable Diffusion.

Primary LanguagePython

DiffusedTexture: AI-Powered Texture Generation for Blender

DiffusedTexture is a Blender add-on that uses Stable Diffusion to create textures directly on 3D meshes.

https://www.cgtrader.com/free-3d-print-models/miniatures/other/elephant-natural-history-museum-1 https://graphics.stanford.edu/data/3Dscanrep/

Table of Contents

Features

  • AI-Driven Texture Creation: Generate diffuse textures directly on 3D models
  • Modes for Different Workflows:
    • Text2Image Parallel: Create textures from text prompts, ensuring global consistency.
    • Image2Image Parallel: Generates textures from input textures, applied parallel across all views.
    • Image2Image Sequential: Sequentially adjusts textures across views, great for refinement.
  • LoRA Integration: Uses LoRA conditioning for specific styles.
  • IPAdapter Integration: Fit specific styles or objects with images for enhanced flexibility and control.

Installation (Windows)

  1. Download 7-Zip
  2. Download all .tar files of the latest release
  3. Untar the file diffused_texture_addon-0.0.1-windows_x64.7z.001 this will automatically untar the other .7z files

    WARNING: DO NOT unzip the resulting diffused_texture_addon-0.0.1-windows_x64.zip

  4. Install the diffused_texture_addon-0.0.1-windows_x64.zip file in Blender as an Add-On.

    • Edit -> Preferences... -> Sidebar Add-ons -> Top right corner dropdown menu -> Install from Disk...

      Installatíon

    • If neccessary, provide a custom HuggingFace Cache Path to install and/or load the checkpoints, else the default path is choosen.

      Download

    • Download necessary models (~10.6 GB total):

      • Tip: Open Blender's system console (Window > Toggle System Console) BEFORE starting the download to monitor download progress.

Installation (Linux)

  • TODO: Test on Linux Instructions and test if it works

Setup

Blender Configuration

  1. Enable CUDA or OPTIX in Blender if using an NVIDIA GPU.
    • Go to Edit > Preferences > System and configure GPU settings.
    • Note: Requires a modern NVIDIA GPU with at least 4GB(-ish) VRAM, 9 and 16 camera parallel runs will require more VRAM.

Usage

General Usage

  1. Load a 3D Model:
    • Import or create a .blend file containing the 3D model.
  2. UV Unwrap the Model:
    • Apply a UV map (Smart UV Project works well).
  3. Access the Add-On:
    • Open the DiffusedTexture panel in the N-panel (right-hand sidebar).
  4. Set Up Texture Generation:
    • Prompt & Negative Prompt: Describe the desired texture/object and what to avoid.
    • Guidance Scale: Adjust creativity vs. fidelity.
    • Denoise Strength: Default to 1.0 for Text2Image.
  5. Adjust Advanced Options:
    • Mesh Complexity:
      • Low: Depth ControlNet only.
      • Medium: Adds Canny ControlNet.
      • High: Adds Normalmap ControlNet for maximum detail.
    • Cameras: Use more viewpoints for better texture blending.
    • Texture & Render Resolution: Ensure render resolution is at least 2x texture resolution.
  6. Generate Texture:
    • Click Start Texture Generation. Monitor progress in the system console.

Additional Options

  • LoRA Models: Add one or multiple LoRA models to match specific results.
  • IPAdapter: Supply the desired "look" as an image instead of a text prompt.

Troubleshooting

  • Add-On Not Visible: Ensure it’s enabled in Edit > Preferences > Add-ons.
  • Blender Freezes: Open the system console to track progress during long tasks.
  • Permission Issues: Specify a valid output path.
  • Out of GPU Memory:
    • Reduce camera count.
    • Close other GPU-intensive applications.
  • Crashes: Restart Blender or your PC if crashes persist.

Roadmap

  • Performance Enhancements:
    • Multi-threaded execution to prevent UI freezing.
  • Checkpoint Flexibility:
    • Allow external Stable Diffusion checkpoints for varied outputs.
  • Masking Support:
    • Apply textures to specific areas of the mesh.
  • Multi-Mesh Workflow:
    • Simultaneously texture multiple objects.

Acknowledgements