/vcnet-blind-image-inpainting

Reproduction of "VCNet: A Robust Approach to Blind Image Inpainting"

Primary LanguagePython

[Re] - VCNet: A Robust Approach to Blind Image Inpainting

This repository is the re-production implementation of VCNet: A Robust Approach to Blind Image Inpainting by Yi Wang, Ying-Cong Chen, Xin Tao and Jiaya Jia in the scope of ML Reproducibility Challenge 2020.

Requirements

To install requirements:

pip install -r requirements.txt

To download datasets, please visit the corresponding web pages: FFHQ - CelebAMaskHQ - Places - ImageNet - graffiti - raindrop.

After downloading, extract each of them to corresponding folder that you created as:

mkdir datasets
cd datasets
mkdir <dataset_name>

Training

To train VCNet from the scratch in the paper, run this command:

python main.py --base_cfg config.yml

Evaluation

To evaluate VCNet on FFHQ with a particular test mode, run:

python main.py --base_cfg config.yml -t -w vcnet.pth --dataset "FFHQ" --test_mode 1

Modes:

  • 1: Contaminant image
  • 2: Random brush strokes with noise
  • 3: Random brush strokes with colors
  • 4: Real occlusions
  • 5: Graffiti
  • 6: Facades (problematic)
  • 7: Words (problematic)
  • 8: Face swapping (problematic)

Pre-trained Models

You can download pretrained models here:

Qualitative Results

From FFHQ-CelebAMaskHQ model (from left to right: the original - the contaminant - broken - gt mask - smoothed mask - predicted mask - output):

From FFHQ-ImageNet model (from left to right: the original - the contaminant - broken - gt mask - smoothed mask - predicted mask - output):

From Places-ImageNet model (from left to right: the original - the contaminant - broken - gt mask - smoothed mask - predicted mask - output):

Quantitative Results

FFHQ

Models BCE PSNR SSIM
Contextual Attention 1.297 16.56 0.5509
GMC 0.766 20.06 0.6675
Partial Conv. 0.400 20.19 0.6795
Gated Conv. 0.660 17.16 0.5915
VCN (original) 0.400 20.94 0.6999
VCN (ours) 0.439 24.76 0.7026

Places2

Models BCE PSNR SSIM
Contextual Attention 0.574 18.12 0.6018
GMC 0.312 20.38 0.6956
Partial Conv. 0.273 19.73 0.6682
Gated Conv. 0.504 18.42 0.6423
VCN (original) 0.253 20.54 0.6988
VCN (ours) 0.437 21.53 0.7070

Contacts