layout2img

This repository includes the implementation for Context-Aware Layout to Image Generation with Enhanced Object Appearance (to appear in CVPR 2021).

This repo is not completely.

Network Structure

Requirements

python3
pytorch >1.0
numpy
matplotlib
opencv

Or install full requirements by running:

pip install -r requirements.txt

TODO

instruction to prepare dataset
remove all unnecessary files
add link to download our pre-trained model
clean code including comments
instruction for training
instruction for evaluation
instruction for applying our methods in layout2img

Training ImageTransformer

Data Preparation

Download COCO dataset to datasets/coco

bash scripts/download_coco.sh

Download VG dataset to datasets/vg

bash scripts/download_vg.sh
python scripts/preprocess_vg.py

Start training

See opts.py for the options.

Evaluation

Trained model

you can download our trained model from our onedrive repo

Performance

You will get the scores close to below after training for around 200 epochs:

models	Resolutions	IS-COCO	IS-VG	FID-COCO	FID-VG
Ours-ED	64*64	15.27+/-.25	8.53+/-.13	31.32	33.91
Ours-D	128*128	15.62+/-.05	12.69+/-.45	22.32	21.78

Reference

If you find this repo helpful, please consider citing:

@inproceedings{he2021context,
  title={Context-Aware Layout to Image Generation with Enhanced Object Appearance},
  author={He, Sen and Liao, Wentong and Yang, Michael and Yang, Yongxin and Song, Yi-Zhe and Rosenhahn, Bodo and Xiang, Tao},
  booktitle={CVPR},
  year={2021}
}

Acknowledgements

This repository is based on LostGAN, and the propsoed modules can be applied in the layout2img.

wtliao/layout2img