Multi-turn Generative Adversarial Networks
- AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks [Paper] [Code]
- StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks [Paper] [Code]
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks [Paper] [Code]
- Generative Adversarial Text-to-Image Synthesis [Paper] Code
- Learning Deep Representations of Fine-grained Visual Descriptions [Paper] Code
- Hardware Setup
- GPU: 2x RTX 2080 Ti
- CPU: 64GB DDR6
- Chipset:
- HDD#1: 1TB SSD
- HDD#2: 4TB SATA
- Software Setup
- PyCharm Installation -
sudo snap install pycharm-community --classic - Tensorflow Installation
- PyTorch Installation
- ElectronJS Installation
- PyCharm Installation -
- Ubuntu Shortcut
- Paste Command - Ctrl + Shift + V
- Windows Screenshot - Alt + PtrScr
- Change Machine Name -
sudo hostnamectl set-hostname AI_Labview change -hostnamectl
python 2.7
Pytorch
In addition, please add the project folder to PYTHONPATH and pip install the following packages:
python-dateutileasydictpandastorchfilenltkscikit-image
Data
- Download our preprocessed metadata for birds coco and save them to
data/ - Download the birds image data. Extract them to
data/birds/ - Download coco dataset and extract the images to
data/coco/
Training
-
Pre-train DMS models:
- For bird dataset:
python pretrain_DMS.py --cfg cfg/DMS/bird.yml --gpu 0 - For coco dataset:
python pretrain_DMS.py --cfg cfg/DMS/coco.yml --gpu 0
- For bird dataset:
-
Train AttnGAN models:
- For bird dataset:
python main.py --cfg cfg/bird_mt.yml --gpu 0 - For coco dataset:
python main.py --cfg cfg/coco_mt.yml --gpu 0
- For bird dataset:
-
*.ymlfiles are example configuration files for training/evaluation our models.
Pretrained Model
-
DMS for bird. Download and save it to
DMSencoders/ -
DMS for coco. Download and save it to
DMSencoders/ -
AttnGAN for bird. Download and save it to
models/ -
AttnGAN for coco. Download and save it to
models/ -
AttnDCGAN for bird. Download and save it to
models/- This is an variant of AttnGAN which applies the propsoed attention mechanisms to DCGAN framework.
Sampling
- Run
python main.py --cfg cfg/eval_bird.yml --gpu 0to generate examples from captions in files listed in "./data/birds/example_filenames.txt". Results are saved toDMSencoders/. - Change the
eval_*.ymlfiles to generate images from other pre-trained models. - Input your own sentence in "./data/birds/example_captions.txt" if you wannt to generate images from customized sentences.
Validation
- To generate images for all captions in the validation dataset, change B_VALIDATION to True in the eval_*.yml. and then run
python main.py --cfg cfg/eval_bird.yml --gpu 0 - We compute inception score for models trained on birds using StackGAN-inception-model.
- We compute inception score for models trained on coco using improved-gan/inception_score.