dalle-playground: A Jupyter Notebook repository from zoobot

Text-to-image Playground (fka DALL-E Playground)

A playground for text-to-image enthusiasts using Stable Diffusion V2.

November 2022, major update

The original version of this repository used DALL-E Mini. With the recent release of Stable Diffusion (SD) V2 and the ease of implementation - this repository has moved to use SD over DALL-E Mini.

Also see Whisper Playground - a playground for building real-time speech2text web apps using OpenAI's Whisper

Fast usage

You can tinker with the DALL-E playground using a Github-hosted frontend. Follow these steps:

Run the DALL-E backend using Google Colab
Copy the URL from the last executed cell. Look for the line having your url is: https://XXXX.trycloudflare.com
Wait for the backend to fully load, this should take ~2min and you should see --> Image generation server is up and running!
Browse https://saharmor.github.io/dalle-playground/?backendUrl=https://XXXX.trycloudflare.com where the backendUrl query parameter should be the url from the previous step

General note: while it is possible to run the backend on the free tier of Google Colab, generating more than ~2 images would take >1min, which will result in a frontend timeout. Consider upgrading to Colab Pro or run the backend notebook on your stronger ML machine (e.g. AWS EC2).

Local development

Follow these steps in case you'd like to clone and run the DALL-E Playground locally:

Clone or fork this repository
Create a virtual environment cd backend && python3 -m venv ENV_NAME
Run virtual environment source venv/bin/activate
Install requirements pip install -r requirements.txt
Make sure you have pytorch and its dependencies installed Installation guide
Run web server python3 app.py --port 8080 --model_version mini (you can change from 8080 to your own port)
In a different terminal, install frontend's modules cd interface && npm install and run it npm start
Copy backend's url from step 5 and paste it in the backend's url input within the web app

Local development/use with Windows WSL2

Window's WSL2 Linux layer has some unique issues getting running with GPU support. Nvidia CUDA drivers are installed on the Windows side instead of Linux, but jax does not see the GPU without compiling from source. Here are extra instructions to get jax compiled.

Have a recent NVIDIA GeForce Game Ready or NVIDIA RTX Quadro driver installed in Windows
In Linux: Install Nvidia's CUDA toolkit, WSL instructions
In Linux: Install Nvidia's CuDNN library: instructions
In Linux: Build and install both jaxlib and jax from source, remember to enable cuda during compilation with python3 build/build.py --enable_cuda instructions
In compiling jaxlib, you might hit a broken configuration file, solution here: jax-ml/jax#11068
Follow local development instructions above

WSL2 installs are fairly bare bones, expect to install packages like npm, python3-pip and many others to get things working. More troubleshooting here

Local development with Docker-compose

Make sure you have docker and The NVIDIA Container Toolkit installed
Clone or fork this repository
start server docker-compose up, add -d to docker-compose up if you'd like to run it in the background
The first time will take some time to download the images, models and other dependencies. models and other dependencies are downloaded only once, and then cached.
Copy backend's url from step 2 and paste it in the backend's url input within the web app.

webapp at http://localhost:3000/dalle-playground

Acknowledgements

The original reposistory used @borisdayma's DALL-E Mini.