/PI-REC

:fire: PI-REC: Progressive Image Reconstruction Network With Edge and Color Domain. :fire: 图像翻译，条件GAN，AI绘画

Primary LanguagePythonOtherNOASSERTION

PI-REC

Version Status Platform PyTorch License Evaluation

Progressive Image Reconstruction Network With Edge and Color Domain

Paper on arXiv | Paper Read Online | BibTex

When I was a schoolchild,

I dreamed about becoming a painter.

With PI-REC, we realize our dream.

For you, for everyone.



English | 中文版


🏳️‍🌈 Demo show time 🏳️‍🌈

Draft2Painting

Tool operation



Introduction

We propose a universal image reconstruction method to represent detailed images purely from binary sparse edge and flat color domain. Here is the open source code and the drawing tool. Learn more about related works here --> image-to-image papers collection.

*The codes of training for release are no completed yet, also waiting for release license of lab.

Find more details in our paper: Paper on arXiv

Quick Overview of Paper

What can we do?

  • Figure (a): Image reconstruction from extreme sparse inputs.
  • Figure (b): Hand drawn draft translation.
  • Figure (c): User-defined edge-to-image (E2I) translation.

Model Architecture

We strongly recommend you to understand our model architecture before running our drawing tool. Refer to the paper for more details.

Prerequisites

  • Python 3+
  • PyTorch 1.0 (0.4 is not supported)
  • NVIDIA GPU + CUDA cuDNN

Installation

  • Clone this repo
  • Install PyTorch and dependencies from http://pytorch.org
  • Install python requirements:
pip install -r requirements.txt

Usage

We provide two ways in this project:

  • Basic command line mode for batch test
  • Drawing tool GUI mode for man-machine interactive creation

Firstly, follow steps below with patience to prepare pre-trained models:

  1. Download the pre-trained models you want here: Google Drive | Baidu (Extraction Code: 9qn1)
  2. Unzip the .7z and put it under your dir ./models/.
    So make sure your path now is: ./models/celeba/<xxxxx.pth>
  3. Complete the above Prerequisites and Installation

Files are ready now! Read the User Manual for firing operations.




中文版介绍 🀄

Demo演示

自己看上面的咯~

简介

我们提出了一种基于GAN的渐进式训练方法 PI-REC，它能从超稀疏二值边缘以及色块中还原重建真实图像。 我们的论文重心是在超稀疏信息输入的还原重建上，并非自动绘画。 总之，PI-REC论文/项目属于图像重建，图像翻译，条件图像生成，AI自动绘画的前沿交叉领域的最新产出，而非简单的以图搜图等等。阅读论文中的 Related Work部分或 image-to-image论文整合项目以了解更多。
注意：这里包含了论文代码以及交互式绘画工具。此论文demo仅推荐给不会绘画的人试玩（比如我），或给予相关领域科研人员参考。远远未达到民用或辅助专业人士绘图的程度。

*由于训练过程过于复杂，用于训练的发布版代码还未完成

在我们的论文中你可以获得更多信息: Paper on arXiv (推荐) | 机器之心-中文新闻稿 | b站中文视频教程(有福利?)

论文概览

PI-REC能做啥？

  • Figure (a): 超稀疏输入信息重建原图。
  • Figure (b): 手绘草图转换。
  • Figure (c): 用户自定义的 edge-to-image (E2I) 转换.

模型结构

我们强烈建议你先仔细阅读论文熟悉我们的模型结构，这会对运行使用大有裨益。

基础环境

  • Python 3
  • PyTorch 1.0 (0.4 会报错)
  • NVIDIA GPU + CUDA cuDNN （当前版本已可选cpu，请修改config.yml中的DEVICE）

第三方库安装

  • Clone this repo
  • 安装PyTorch和torchvision --> http://pytorch.org
  • 安装 python requirements:
pip install -r requirements.txt

运行使用

我们提供以下两种方式运行：

  • 基础命令行模式 用来批处理测试整个文件夹的图片
  • 绘画GUI工具模式 用来实现交互式创作

首先，请耐心地按照以下步骤做准备：

  1. 在这里下载你想要的预训练模型文件：Google Drive | Baidu (提取码: 9qn1)

更新：2021.4 baidu网盘和谐了我的权重文件，已无法分享，请自行前往Google drive~

  1. 解压，放到目录./models下
    现在你的目录应该像这样： ./models/celeba/<xxxxx.pth>
  2. 完成上面的基础环境和第三方库安装

啦啦啦啦，到这里准备工作就完成啦，接下来需要阅读用户手册来运行程序~



Acknowledgment

Code structure is modified from Anime-InPainting, which is based on Edge-Connect.

BibTex 

@article{you2019pirec,
  title={PI-REC: Progressive Image Reconstruction Network With Edge and Color Domain},
  author={You, Sheng and You, Ning and Pan, Minxue},
  journal={arXiv preprint arXiv:1903.10146},
  year={2019}
}