This runs stable diffusion v1.5, grabs images from an active Minecraft Window and runs an Image2Image pipeline using a text description to generate real-time art from the game.
- Windows 11 or 10
- Python 3.9 installed (From the Microsoft store)
- Minecraft Java Edition
- Bandwidth internet connection to download the models
- NVIDIA GPU with CUDA 11.4 and 8gb of vram
Don't meet the requirements?
There are ways to run stable diffuson on AMD Video Cards and even in CPUs without a graphics card, however, I don't plan to support AMD Video Cards nor CPU generations. If you want to try for yourself, just be aware that this script only runs on Windows, and AMD support is mainly focused on Linux. Also, if you try running it on CPU with no graphics card you may have a really bad time waiting for 5 to 10 minutes just to get a frame, so it would be better if you tried another way, like a Google Collab with stable diffusion where you upload your own screenshots. This is mostly intended for advanced users in an effort to point you towards the right direction.- Install Python 3.9 from https://www.python.org/downloads/
- Install the latest NVIDIA drivers from https://www.nvidia.com/Download/index.aspx?lang=en-us
- Install CUDA from https://developer.nvidia.com/cuda-downloads
- Create virtual environment
python -m venv venv
- Activate the virual environment
- If you're on CMD, use
.\venv\Scripts\activate.bat - If you're on PowerShell, use
.\venv\Scripts\Activate.ps1- Keep in mind you should have enabled scripts in Windows PS
- If you're on CMD, use
- Install the requirements
pip install -r requirements.txt
- Open Minecraft and set the resolution to 768x768
- Run the script
python diffusioncraft.py
You may play with the strenght and the camera distance from the objects.
You will find all the images you generated inside the tmp folder with the name of the seed you used.
Some good prompts are:
An isometric drawing of a house, concept art, trending on artstationcrochet bunny with a scarfcinematic mountainscape highly detailed, epic, with old town by greg rutkowskicolourful watercolor of astronaut in space surrounded by planets and starsgod prince face close-up portrait ram skull abstract 3d composition. jellyfish phoenix head, nautilus, orchid, skull, betta fish, bioluminiscent creatures, intricate artwork by Tooth Wu and wlop and beeple. octane render, trending on artstation, greg rutkowski very coherent symmetrical artwork. cinematic, hyper realism, high detail, octane render, 8krealistic photo of a human face, portrait, sketchdetailed photo of a fan-palm plant in a junglescapepanoramic photo of a neon cyberpunk city skyline, detailed, high resolution, infocusoctane render of a cute retro robota photorealistic handa beautiful storybook painting of a vintage Schwinn bicycle with a basket of flowers on the front propped up against a wall, anime style by Grzegorz greg rutkowski and Studio Ghibli, nostalgic heart-warming, trending on artstation hqrealistic skull with horns, portrait, powerful, intricate, elegant, volumetric lighting, scenery, digital painting, highly detailed, artstation, sharp focus, illustration, concept art, ruan jia, steve mccurryHD nature photography of ladybugisometric photo of a realistic sunlit cozy refreshing outdoor summer reflective indoors poolside lounge, 3d octane render, isometric anglephoto of a sock puppet, muppet, highly detailed, 4kcomplex 3 d render hyper detail portrait of a mechanical cyborg robot, sci fi, full body, intricate, art by kazuhiko nakamura and hajime sorayama, 8 k octane detailed render, post processing, extremely hyperdetailed, intricate futuristic mechanic parts, maya, dark background, sharp focus, blender, cinematic lighting masterpiece, trending on artstationrealistic full body, Portrait painting of Tom Hiddleston as Cyborg Power Ranger, made by Michaelangelo, physical painting, Sharp focus,digital art, bright colors,fine art, trending on Artstation, unreal engine
Original Idea by: ThoseSixFaces
Funding: BobicraftMC