This technical demo is an open-source project that allows users to customize the appearance and design of the map in game with stable diffusion. You can modify graphical elements to create your own unique version map.
full video demo here : https://youtu.be/vJml7Hf4jPs
full animation video demo here : https://www.youtube.com/watch?v=dwrt-kQ-xl4
- Create your own map with stable diffusion
- Generate animations for specific elements of the map
To run this project, make sure you have the following installed on your machine:
- Python 3.10 or higher -
- Pygame 2.0.0 or higher
- pygame-widgets 1.0.0
- Pillow 9.4.0
- requests 2.2.25
- Stable diffusion webui automatic1111
You can install Stable Diffusion Webui by following the instructions on the Stable Diffusion Webui repository.
To install the game, follow these steps:
-
Launch Stable diffusion webui with "--api" flag
-
Clone this repository to your local directory:
git clone https://github.com/numz/StableDiffusionPygameInpaintIsometricMap.git- Navigate to the project directory:
cd StableDiffusionPygameInpaintIsometricMap- Install the required dependencies:
pip install -r requirements.txt- Launch the game using the following command:
python main.py- Draw a Shape by maintain left shift key + left mouse click to draw your shape (minimum 3 points for see shape. Shape color will be red),
- Once shape finish release left shift key, prompt input will appear, write your prompt and press enter to validate.
- First, you need to create an element to animate (see "Create elements" section)
- Draw a Shape by maintain left Ctrl key + left mouse click to draw your shape (minimum 3 points for see shape. Shape color will be blue),
- Once shape finish release left Ctrl key, prompt input will appear, write your prompt and press enter to validate.
Note that the prompt does not have much influence on the outcome, but it is still recommended to have a prompt closely related to the description of the object being addressed. For example: If you want to animate water, include 'water' in the prompt.
You can animate a lot of kind of elements, for example :
- water
- fire
- lava
- smoke
- clouds
- grass
- trees
- etc...
the file Config/config.py contains parameters of the game, especially the prompt. here is the default prompt use for demo :
- "3d render, isometric %s, octane render, by greg rutkowski"
- "%s" will be replaced by your prompt input in game
models use in demo : v1-5-pruned-emaonly.ckpt
in the "payloads" folder, "payload.json" contain payload used for stable diffusion img2img endpoint, feel free to edit.
- Press "Escape" : access the menu
- Draw a Shape for inpainting : maintain left shift key + left click to draw your shape (minimum 3 points for see shape)
- Draw a Shape for animation : maintain left ctrl key + left click to draw your shape (minimum 3 points for see shape)
- Enter : validate your prompt
- arrow keys : move on the map
- "+" key : speed up time
- "-" key : slow down time
- Add key control for stable diffusion parameters
- Create Tile editor
Contributions are welcome! To contribute, please follow these steps:
- Fork this repository to your GitHub account.
- Clone the forked repository to your local machine.
- Create a new branch for your changes.
- Make your changes and commit them.
- Push the branch to your forked repository.
- Create a pull request on this original repository to submit your changes.
This project is licensed under the MIT License. For more information, see the LICENSE file.