!!! Outdated please use this improved repro !!!
A slightly modified version of InvokeAI: A Stable Diffusion Toolkit / CompVis/stable-diffusion where the stable diffusion model can be connected to grasshopper (or anything else) through sockets. The preset mode is image2image translation where a prompt is combined with a guiding input image. In this case, the guiding image is taken from rhino's viewport. Additionally, a simple humanUI interface is provided. Moreover, the script calling the stable diffusion model through a socket is wrapped in a hops definition to utilize its async functionality (while the image is computed, the viewport won't freeze).
A) Stable Diffusion Model
- Clone this repro and follow Installation instructions
B) Grasshopper Plugins
-
Fulfill Prerequisites
-
Download this repository
-
replace dream.py file: in your stable diffusion folder (prerequisites Step A), navigate to \scripts and replace the dream.py file with the dream.py file from this repro
-
Open Rhino 7+, launch grasshopper and open main.gh. In the upper section of the humanUI window, set viewport name and file path to the stable diffusion model
-
open an anaconda prompt and paste the commands to start the stable diffusion model (see image below)
Sample Images
Naturally, the model is not limited to a particular domain