This repository contains our official implementation of the CVPR 2024 paper: SVGDreamer: Text-Guided SVG Generation with Diffusion Model. It can generate high-quality SVGs based on text prompts.
- [03/2024] 🔥 We have released the code for SVGDreamer.
- [02/2024] 🎉 SVGDreamer accepted by CVPR2024. 🎉
- [12/2023] 🔥 We have released the SVGDreamer Paper. SVGDreamer is a novel text-guided vector graphics synthesis method. This method considers both the editing of vector graphics and the quality of the synthesis.
You can follow the steps below to quickly get up and running with SVGDreamer. These steps will let you run quick inference locally.
In the top level directory run,
sh script/install.shor using docker,
docker run --name svgdreamer --gpus all -it --ipc=host ximingxing/svgrender:v1 /bin/bashDownloading pretrained SD models by setting diffuser.download=True in /conf/config.yaml the first time you run
it.
(Alternatively, you can append diffuser.download=True to the end of the script.)
Or you can still download it manually,
- Model Link: https://huggingface.co/stabilityai/stable-diffusion-2-1-base
- Default model is stored in the
/home/user/.cache/huggingface/hub/models--stabilityai--stable-diffusion-2-1-base
Prompt: an image of Batman. full body action pose, complete detailed body, white background, high quality, 4K, ultra
realistic
Preview:
| Particle 1 | Particle 2 | Particle 3 | Particle 4 | Particle 5 | Particle 6 |
|---|---|---|---|---|---|
| init p1 | init p2 | init p3 | init p4 | init p5 | init p6 |
 |
 |
 |
 |
 |
 |
| final p1 | final p2 | final p3 | final p4 | final p5 | final p6 |
 |
 |
 |
 |
 |
 |
Script:
python svgdreamer.py x=iconography skip_sive=False "prompt='an image of Batman. full body action pose, complete detailed body. white background. empty background, high quality, 4K, ultra realistic'" token_ind=4 x.vpsd.t_schedule='randint' result_path='./logs/batman' multirun=Truex=iconography(str): style configsskip_sive(bool): enable the SIVE stagetoken_ind(int): the index of text prompt, from 1result_path(str): the path to save the resultmultirun(bool): run the script multiple times with different random seedsmv(bool): save the intermediate results of the run and record the video (This increases the run time)
More parameters in ./conf/x/style.yaml, you can modify these parameters from the command line. For example,
append x.vpsd.n_particle=4 to the end of the script.
Prompt: an astronaut walking across a desert, planet mars in the background, floating beside planets, space
art
Preview:
| attn-map | bg init | fg init | bg final | fg final | final |
|---|---|---|---|---|---|
 |
 |
 |
 |
 |
 |
Script:
python svgdreamer.py x=iconography_s1 skip_sive=False "prompt='a man in an astronaut suit walking across a desert, inspired by James Gurney, space art, planet mars in the background, banner, floating beside planets'" token_ind=5 x.vpsd.t_schedule='randint' result_path='./logs/astronaut_sive' multirun=TruePrompt: Sydney opera house. oil painting. by Van Gogh
Preview:
| Particle 1 | Particle 2 | Particle 3 | Particle 4 | Particle 5 | Particle 6 |
|---|---|---|---|---|---|
| init p1 | init p2 | init p3 | init p4 | init p5 | init p6 |
 |
 |
 |
 |
 |
 |
| final p1 | final p2 | final p3 | final p4 | final p5 | final p6 |
 |
 |
 |
 |
 |
 |
Script:
python svgdreamer.py x=iconography "prompt='Sydney opera house. oil painting. by Van Gogh'" result_path='./logs/SydneyOperaHouse-OilPainting'Prompt: Abstract Vincent van Gogh Oil Painting Elephant, featuring earthy tones of green and brown
Preview:
| Particle 1 | Particle 2 | Particle 3 | Particle 4 | Particle 5 | Particle 6 |
|---|---|---|---|---|---|
| init p1 | init p2 | init p3 | init p4 | init p5 | init p6 |
 |
 |
 |
 |
 |
 |
| final p1 | final p2 | final p3 | final p4 | final p5 | final p6 |
 |
 |
 |
 |
 |
 |
Script:
python svgdreamer.py x=painting "prompt='Abstract Vincent van Gogh Oil Painting Elephant, featuring earthy tones of green and brown.'" x.num_paths=256 result_path='./logs/Elephant-OilPainting'Prompt: Darth vader with lightsaber
Preview:
| Particle 1 | Particle 2 | Particle 3 | Particle 4 | Particle 5 | Particle 6 |
|---|---|---|---|---|---|
| init p1 | init p2 | init p3 | init p4 | init p5 | init p6 |
 |
 |
 |
 |
 |
 |
| final p1 | final p2 | final p3 | final p4 | final p5 | final p6 |
 |
 |
 |
 |
 |
 |
Script:
python svgdreamer.py x=pixelart "prompt='Darth vader with lightsaber.'" result_path='./logs/DarthVader'Prompt: A picture of a bald eagle. low-ploy. polygon. minimal flat 2d vector
Preview:
| Particle 1 | Particle 2 | Particle 3 | Particle 4 | Particle 5 | Particle 6 |
|---|---|---|---|---|---|
| init p1 | init p2 | init p3 | init p4 | init p5 | init p6 |
 |
 |
 |
 |
 |
 |
| final p1 | final p2 | final p3 | final p4 | final p5 | final p6 |
 |
 |
 |
 |
 |
 |
Script:
python svgdreamer.py x=lowpoly "prompt='A picture of a bald eagle. low-ploy. polygon. minimal flat 2d vector'" neg_prompt='' result_path='./logs/BaldEagle'Prompt: A free-hand drawing of A speeding Lamborghini. black and white drawing.
Preview:
| Particle 1 | Particle 2 | Particle 3 | Particle 4 | Particle 5 | Particle 6 |
|---|---|---|---|---|---|
| init p1 | init p2 | init p3 | init p4 | init p5 | init p6 |
 |
 |
 |
 |
 |
 |
| final p1 | final p2 | final p3 | final p4 | final p5 | final p6 |
 |
 |
 |
 |
 |
 |
Script:
python svgdreamer.py x=sketch "prompt='A free-hand drawing of A speeding Lamborghini. black and white drawing.'" neg_prompt='' result_path='./logs/Lamborghini'Prompt: Big Wild Goose Pagoda. ink style. Minimalist abstract art grayscale watercolor. empty background
Preview:
| Particle 1 | Particle 2 | Particle 3 | Particle 4 | Particle 5 | Particle 6 |
|---|---|---|---|---|---|
| init p1 | init p2 | init p3 | init p4 | init p5 | init p6 |
 |
 |
 |
 |
 |
 |
| final p1 | final p2 | final p3 | final p4 | final p5 | final p6 |
 |
 |
 |
 |
 |
 |
Script:
python svgdreamer.py x=ink "prompt='Big Wild Goose Pagoda. ink style. Minimalist abstract art grayscale watercolor. empty background'" neg_prompt='' result_path='./logs/BigWildGoosePagoda'See Examples.md for more cases.
- I highly recommend turning on xformer
enable_xformers=Trueto speed up optimization. x.vpsd.t_schedulegreatly affects the style of the result. Please try more.neg_promptnegative prompts affect the quality of the results- By setting
state.mprec='fp16', you can significantly reduce GPU memory usage.
- Release the code.
- Add docker image.
- Support fp16 optimization.
The project is built based on the following repository:
- BachiLi/diffvg
- huggingface/diffusers
- ximinng/DiffSketcher
- THUDM/ImageReward
- ximinng//PyTorch-SVGRender
We gratefully thank the authors for their wonderful works.
If you use this code for your research, please cite the following work:
@article{xing2023svgdreamer,
title={SVGDreamer: Text Guided SVG Generation with Diffusion Model},
author={Xing, Ximing and Zhou, Haitao and Wang, Chuang and Zhang, Jing and Xu, Dong and Yu, Qian},
journal={arXiv preprint arXiv:2312.16476},
year={2023}
}
This work is licensed under a MIT License.