/ZeDuSR

Zero-Shot Dual-Lens Super-Resolution, CVPR2023

Primary LanguagePython

Zero-Shot Dual-Lens Super-Resolution

Zero-Shot Dual-Lens Super-Resolution, In CVPR 2023. Ruikang Xu, Mingde Yao, Zhiwei Xiong.

Paper|Slides|Video

Datesets

Real-world Dataset

  • The CameraFusion (by iPhone11) dataset can be downloaded from this link.
  • The RealMCVSR (by iPhone12) dataset can be downloaded from this link.

Synthesized Dataset

  • The HCI_new dataset can be downloaded from this link.
  • The Middlebury2021 dataset can be downloaded from this link.
  • Other stereo image datasets (Flick1024, Holopix50k) and light-field datasets (NTIRE2023, Stanford) also can be used.

Dependencies

  • Python 3.8.8, PyTorch 1.8.0, torchvision 0.9.0.
  • NumPy 1.24.2, OpenCV 4.7.0, Tensorboardx 2.5.1, Pillow, Imageio.

Data Pre-processing

Dependencies

Please follow README files in the corresponding subfolders to bulid environments for runing.

Real-world Dataset

We take wide-angle images SR with telephoto images on iphone11 as example.

  • Pre-Alignment: 
    cd ./RealworldData/PreAlignment && sh run.sh
  • Color correction: 
    cd ./RealworldData/color_correction

mkdir ./Data/TeleView_SIFTAlign_cor

run ColorIuminanceDir.m

Synthesized Dataset

We take 2x downsampling using isotropic Gaussian kernel with slight JPEG compression as example.

  • Synthesized Data Generation:

    cd ./SynthesizedData/DataGeneration && sh run.sh

  • Pre-Alignment:

    cd ./SynthesizedData/PreAlignment && sh run.sh

Quick Start

Real-world Dataset

We take wide-angle images SR with telephoto images on iphone11 as example.

  • Degradation‐invariant Alignment for Dual-lens Images: 
    cd ./Alignment && sh align_real.sh
  • ZSSR with Degradation‐aware Training and Inference: 
    cd ./SR && sh sr_real.sh

Synthesized Dataset

We take 2x downsampling using isotropic Gaussian kernel with slight JPEG compression as example.

  • Degradation‐invariant Alignment for Dual-lens Images: 
    cd ./Alignment && sh align_synth.sh
  • ZSSR with Degradation‐aware Training and Inference: 
    cd ./SR && sh sr_synth.sh

Contact

Any question regarding this work can be addressed to xurk@mail.ustc.edu.cn and mdyao@mail.ustc.edu.cn.

Citation

If you find our work helpful, please cite the following paper.

@InProceedings{Xu_2023_CVPR,
    author    = {Xu, Ruikang and Yao, Mingde and Xiong, Zhiwei},
    title     = {Zero-Shot Dual-Lens Super-Resolution},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    year      = {2023}
}