Haoming02/stable-diffusion-for-unity

A project that allows you to run Stable Diffusion NATIVELY inside Unity at runtime

No Longer Maintained

This project was originally just a proof of concept. With so many new optimization techniques, such as Turbo, this implementation is extremely slow by today's standard...

Stable Diffusion for Unity

This is an Unity project that allows you to run the Stable Diffusion pipeline within Unity, achieved by using ONNX version of the models and OnnxRuntime to inference at runtime.

Requirements

• CUDA Toolkit
• CUDA Deep Neural Network
• ONNX Runtime

  as shown in the Plugins folder

Supported Versions

• This project was built on ONNX Runtime 1.15.1
  • Tested on CUDA v11.6 and v11.8
  • Tested on CUDNN v8.5.0.96

Getting Started

• Download the models from Release, and put them into the StreamingAssets folder.
• 2 sample scenes are included:
  • One is a simple UI where you can enter any prompt and adjust some settings to generate an image
  • Another is a 3D scene where you can generate a skybox and a painting

Implemented Features

• Positive Prompt
• Steps
• Euler a sampling method
• CFG Scale
• Inference fp16 Precision

References

• https://onnxruntime.ai/docs/tutorials/csharp/stable-diffusion-csharp.html
• https://github.com/cassiebreviu/StableDiffusion
• https://onnxruntime.ai/docs/get-started/with-csharp.html
• https://onnxruntime.ai/docs/tutorials/csharp/csharp-gpu.html