Spatially-Adaptive Feature Modulation for Efficient Image Super-Resolution

Long Sun, Jiangxin Dong, Jinhui Tang, Jinshan Pan

IMAG Lab, Nanjing University of Science and Technology

An overview of the proposed SAFMN. SAFMN first transforms the input LR image into the feature space using a convolutional layer, performs feature extraction using a series of feature mixing modules (FMMs), and then reconstructs these extracted features by an upsampler module. The FMM block is mainly implemented by a spatially-adaptive feature modulation (SAFM) layer and a convolutional channel mixer (CCM).

News

[2023-04-03] The SAFMN variant ranks Top3 for model complexity in the NTIRE2023 ESR Challenge.
[2023-03-22] The code and checkpoint for the NTIRE2023 Efficient Super-Resolution Challenge is available here.
[2023-03-22] The pre-trained model with high-order degradation on the LSDIR dataset is available.
[2023-03-13] The source codes, checkpoints and visual results are available.
[2023-02-26] The paper is available here.

Requirements

Python 3.8, PyTorch >= 1.11

BasicSR 1.4.2

Platforms: Ubuntu 18.04, cuda-11

Installation

# Clone the repo
git clone https://github.com/sunny2109/SAFMN.git
# Install dependent packages
cd SAFMN
pip install -r requirements.txt
# Install BasicSR
python setup.py develop

You can also refer to this INSTALL.md for installation

Training

Run the following commands for training:

# train SAFMN for x4 effieicnt SR
python basicsr/train.py -opt options/train/SAFMN/train_DF2K_x4.yml
# train SAFMN for x4 classic SR
python basicsr/train.py -opt options/train/SAFMN/train_L_DF2K_x4.yml

Testing

Download the pretrained models.
Download the testing dataset.
Run the following commands:

# test SAFMN for x4 efficient SR
python basicsr/test.py -opt options/test/SAFMN/test_benchmark_x4.yml
# test SAFMN for x4 classic SR
python basicsr/test.py -opt options/test/SAFMN/test_L_benchmark_x4.yml
# test SAFMN for x4 real-world SR (without ground-truth)
python basicsr/test.py -opt options/test/SAFMN/test_real_img_x4.yml

The test results will be in './results'.

Results

Pretrained models and visual results

Degradation	Model Zoo	Visual Results
BI-Efficient SR	Google Drive/Baidu Netdisk with code: SAFM	Google Drive/Baidu Netdisk with code: SAFM
BI-Classic SR	Google Drive/Baidu Netdisk with code: SAFM	Google Drive/Baidu Netdisk with code: SAFM
x4 High-order	Google Drive/Baidu Netdisk with code: SAFM

Efficient SR Results

Classic SR Results

Real-world SR Results

Real-World Image (x4)	Real-ESRGAN	SwinIR	SAFMN (ours)

Runtime Comparison

Comparison with NTIRE Winners

Citation

If this work is helpful for your research, please consider citing the following BibTeX entry.

@article{sun2023safmn,
    title={Spatially-Adaptive Feature Modulation for Efficient Image Super-Resolution},
    author={Sun, Long and Dong, Jiangxin and Tang, Jinhui and Pan, Jinshan},
    journal={arXiv preprint arXiv:2302.13800},
    year={2023}
 }

Acknowledgement

This code is based on BasicSR toolbox. Thanks for the awesome work.

wwhio/SAFMN