A playground for DALL-E enthusiasts to tinker with the open-source version of OpenAI's DALL-E, based on DALL-E Mini.
DALLE.Playground.demo.mp4
This is a WIP based on an older version of this repo, if I have time I will update it to the current version.
Added app_8GB_VRAM.py to the backend folder.
I needed to run with XLA_FLAGS='--xla_gpu_force_compilation_parallelism=1 --xla_gpu_strict_conv_algorithm_picker=false'
as well as XLA_PYTHON_CLIENT_ALLOCATOR=platform
This was tested on a 8GB RTX 3060ti - it fits the f16 mega version by unloading and loading VQGAN when generating images. It is slower, but not too much, since the model is only downloaded once.
I have also removed the wandb call in this version, but that shouldn't matter.
You can tinker with the DALL-E playground using a Github-hosted frontend. Follow these steps:
- Run the DALL-E backend using Google Colab
- Copy the URL from the last executed cell. Look for the line having
your url is: https://XXXX.loca.lt
- Wait for the backend to fully load, this should take ~2min and you should see
--> DALL-E Server is up and running!
- Browse https://saharmor.github.io/dalle-playground/?backendUrl=https://XXXX.loca.lt where the
backendUrl
query parameter should be the url from the previous step
General note: while it is possible to run the DALL-E Mini backend on the free tier of Google Colab, generating more than 1-2 images would take more than 1min, which will result in a frontend timeout. Consider upgrading to Colab Pro or run the backend notebook on your stronger ML machine (e.g. AWS EC2).
DALL-E Mega is substianlly more capable than DALL-E Mini and therefore generates higher fidelity images. If you have the computing power--either through a Google Colab Pro+ subcription or by having a strong local machine, uncomment this line before running the backend.
Follow these steps in case you'd like to clone and run the DALL-E playground locally:
- Clone or fork this repository
- Create a virtual environment
cd backend && python3 -m venv ENV_NAME
- Install requirements
pip install -r requirements.txt
- Make sure you have pytorch and its dependencies installed Installation guide
- Run web server
python app.py 8080
(you can change from 8080 to your own port) - In a different terminal, install frontend's modules
cd interface && npm install
and run itnpm start
- Copy backend's url from step 5 and paste it in the backend's url input within the web app
-
Make sure you have docker and The NVIDIA Container Toolkit installed
-
Clone or fork this repository
-
start server
docker-compose up
, add-d
todocker-compose up
if you'd like to run it in the background -
The first time will take some time to download the images, models and other dependencies. models and other dependencies are downloaded only once, and then cached.
-
Copy backend's url from step 2 and paste it in the backend's url input within the web app.
webapp at
http://localhost:3000/dalle-playground
This repo is a full-stack flavour of Boris Dayma's DALL-E Mini repository.