Paint Transformer: Feed Forward Neural Painting with Stroke Prediction

[Paper] [Paddle Implementation]

Update

We have optimized the serial inference procedure to achieve better rendering quality and faster speed.

Overview

This repository contains the official PaddlePaddle implementation of paper:

Paint Transformer: Feed Forward Neural Painting with Stroke Prediction,

Songhua Liu*, Tianwei Lin*, Dongliang He, Fu Li, Ruifeng Deng, Xin Li, Errui Ding, Hao Wang (* indicates equal contribution)

ICCV 2021 (Oral)

Prerequisites

Linux or macOS
Python 3.6+
PaddlePaddle 2.0+ and other dependencies (numpy, cv2, and other common python libs)
```
python -m pip install paddlepaddle-gpu
```

Getting Started

Clone this repository:

git clone https://github.com/wzmsltw/PaintTransformer
cd PaintTransformer

Download pretrained model from Google Drive and move it to inference directory:
```
mv [Download Directory]/paint_best.pdparams inference/
cd inference
```
Inference:
```
python inference.py
```
- Input image path, output path, and etc can be set in the main function.
- Notably, there is a flag serial as one parameter of the main function:
  - If serial is True, strokes would be rendered serially. The consumption of video memory will be low but it requires more time. Serial inference can achieve better rendering quality.
  - If serial is False, strokes would be rendered in parallel. The consumption of video memory will be high but it would be faster.
  - If animated results are required, serial must be True.
Train:
- You can send email to us for the training codes.

More Results

Input	Animated Output

App

Do not want to run the code? Try an App 一刻相册 downloaded from here!

Citation

If you find ideas or codes useful for your research, please cite:

@inproceedings{liu2021paint,
  title={Paint Transformer: Feed Forward Neural Painting with Stroke Prediction},
  author={Liu, Songhua and Lin, Tianwei and He, Dongliang and Li, Fu and Deng, Ruifeng and Li, Xin and Ding, Errui and Wang, Hao},
  booktitle={Proceedings of the IEEE International Conference on Computer Vision},
  year={2021}
}

Contact

For any question, please file an issue or contact

Songhua Liu: songhua.liu@smail.nju.edu.cn
Tianwei Lin: lintianwei01@baidu.com