Deep Cyclic Generative Adversarial Residual Convolutional Networks for Real Image Super-Resolution (SRResCycGAN)

An official PyTorch implementation of the SRResCycGAN network as described in the paper Deep Cyclic Generative Adversarial Residual Convolutional Networks for Real Image Super-Resolution. This work is participated in the AIM 2020 Real-Image Super-resolution challenge track-3 at the high x4 upscaling factor.

Abstract

Recent deep learning based single image super-resolution (SISR) methods mostly train their models in a clean data domain where the low-resolution (LR) and the high-resolution (HR) images come from noise-free settings (same domain) due to the bicubic down-sampling assumption. However, such degradation process is not available in real-world settings. We consider a deep cyclic network structure to maintain the domain consistency between the LR and HR data distributions, which is inspired by the recent success of CycleGAN in the image-to-image translation applications. We propose the Super-Resolution Residual Cyclic Generative Adversarial Network (SRResCycGAN) by training with a generative adversarial network (GAN) framework for the LR to HR domain translation in an end-to-end manner. We demonstrate our proposed approach in the quantitative and qualitative experiments that generalize well to the real image super-resolution and it is easy to deploy for the mobile/embedded devices. In addition, our SR results on the AIM 2020 Real Image SR Challenge datasets demonstrate that the proposed SR approach achieves comparable results as the other state-of-art methods.

Spotlight Video

Pre-trained Models

Datasets	SRResCycGAN
NTIRE2020 RWSR	Sensor noise (σ = 8)
NTIRE2020 RWSR	JPEG compression (quality=30)
NTIRE2020 RWSR	Unknown corruptions
AIM2020 RISR	Real image corruptions

BibTeX

@InProceedings{Umer_2020_ECCVW,
               author = {Muhammad Umer, Rao and Micheloni, Christian},
               title = {Deep Cyclic Generative Adversarial Residual Convolutional Networks for Real Image Super-Resolution},
               booktitle = {Proceedings of the European Conference on Computer Vision (ECCV) Workshops},
               month = {August},
               year = {2020}
              }

Quick Test

This model can be run on arbitrary images with a Docker image hosted on Replicate: https://beta.replicate.ai/RaoUmer/SRResCycGAN. Below are instructions for how to run the model without Docker:

Dependencies

Python 3.7 (version >= 3.0)
PyTorch >= 1.0 (CUDA version >= 8.0 if installing with CUDA.)
Python packages: pip install numpy opencv-python

Train models

The SR training code is based on the SRResCGAN.

Test models

Clone this github repository as the following commands:

git clone https://github.com/RaoUmer/SRResCycGAN
cd SRResCycGAN
cd srrescycgan_code_demo

Place your own low-resolution images in the ./srrescycgan_code_demo/LR folder. (There are two sample images i.e. LR_006 and LR_014).
Download the pretrained models from Pre-trained Models section. Place the models in ./srrescycgan_code_demo/trained_nets_x4.
Run the test. You can config in the test_srrescycgan.py.

python test_srrescgan.py

The results are in the ./srrescycgan_code_demo/sr_results_x4 folder.

SRResCycGAN Architecture

Overall Representative diagram

Quantitative Results

The x4 SR quantitative results comparison of our method with others over the DIV2K validation-set (100 images). The best performance is shown in red and the second best performance is shown in blue.

The AIM2020 Real Image SR Challenge Results (x4)

_Team	_PSNR↑	_SSIM↑	_{Weighed_score↑}
_Baidu	_31.3960	_0.8751	_{0.7099 (1)}
_ALONG	_31.2369	_0.8742	_{0.7076 (2)}
_CETC-CSKT	_31.1226	_0.8744	_{0.7066 (3)}
_SR-IM	_31.2369	_0.8728	_0.7057
_DeepBlueAI	_30.9638	_0.8737	_0.7044
_JNSR	_30.9988	_0.8722	_0.7035
_{OPPO_CAMERA}	_30.8603	_0.8736	_0.7033
_Kailos	_30.8659	_0.8734	_0.7031
_{SR_DL}	_30.6045	_0.8660	_0.6944
_{Noah_TerminalVision}	_30.5870	_0.8662	_0.6944
_Webbzhou	_30.4174	_0.8673	_0.6936
_{TeamInception}	_30.3465	_0.8681	_0.6935
_IyI	_30.3191	_0.8655	_0.6911
_MCML-Yonsei	_30.4201	_0.8637	_0.6906
_MoonCloud	_30.2827	_0.8644	_0.6898
_qwq	_29.5878	_0.8547	_0.6748
_SrDance	_29.5952	_0.8523	_0.6729
_{MLP_SR (ours)}	_28.6185	_0.8314	_0.6457
_EDSR	_28.2120	_0.8240	_0.6356
_{RRDN_IITKGP}	_27.9708	_0.8085	_0.6201
_congxiaofeng	_26.3915	_0.8258	_0.6187

Visual Results

DIV2K Validation-set (100 images)

Here are the SR resutls comparison of our method on the DIV2K validation-set images.

Real-Image SR Challenge dataset images (Track-3)

Validation-set

You can download all the SR resutls of our method on the AIM 2020 Real-Image SR validation-set from the Google Drive: SRResCycGAN.

Test-set

You can download all the SR resutls of our method on the AIM 2020 Real-Image SR test-set from the Google Drive: SRResCycGAN.