DF-GAN: Deep Fusion Generative Adversarial Networks for Text-to-Image Synthesis
Official Pytorch implementation for our paper DF-GAN: Deep Fusion Generative Adversarial Networks for Text-to-Image Synthesis by Ming Tao, Hao Tang, Songsong Wu, Nicu Sebe, Fei Wu, Xiao-Yuan Jing.
Requirements
- python 3.6+
- Pytorch 1.0+
- easydict
- nltk
- scikit-image
- A titan xp (set nf=32 in *.yaml) or a V100 32GB (set nf=64 in *.yaml)
Installation
Clone this repo.
git clone https://github.com/tobran/DF-GAN
cd DF-GAN/code/
Datasets Preparation
- Download the preprocessed metadata for birds coco and save them to
data/
- Download the birds image data. Extract them to
data/birds/
- Download coco dataset and extract the images to
data/coco/
Pre-trained text encoder
- Download the pre-trained text encoder for CUB and save it to
DAMSMencoders/bird/inception/
- Download the pre-trained text encoder for coco and save it to
DAMSMencoders/coco/inception/
Training
Train DF-GAN models:
-
For bird dataset:
python main.py --cfg cfg/bird.yml
-
For coco dataset:
python main.py --cfg cfg/coco.yml
-
*.yml
files are example configuration files for training/evaluation our models.
Evaluating
Dwonload Pretrained Model
- DF-GAN for bird. Download and save it to
models/bird/
- DF-GAN for coco. Download and save it to
models/coco/
Evaluate DF-GAN models:
- To evaluate our DF-GAN on CUB, change B_VALIDATION to True in the bird.yml. and then run
python main.py --cfg cfg/bird.yml
- To evaluate our DF-GAN on coco, change B_VALIDATION to True in the coco.yml. and then run
python main.py --cfg cfg/coco.yml
- We compute inception score for models trained on birds using StackGAN-inception-model.
- We compute FID for CUB and coco using DM-GAN/eval/FID.
Citing DF-GAN
If you find DF-GAN useful in your research, please consider citing our paper:
@article{ming2020DFGAN,
title={DF-GAN: Deep Fusion Generative Adversarial Networks for Text-to-Image Synthesis},
author={Ming Tao, Hao Tang, Songsong Wu, Nicu Sebe, Fei Wu, Xiao-Yuan Jing},
journal={arXiv preprint arXiv:2008.05865},
year={2020}
}
The code is released for academic research use only. For commercial use, please contact Ming Tao.
Reference