Self-Convolution is a self-supervised and highly-efficient image operator that exploits non-local similarity. Self-Convolution can generalize many commonly-used non-local schemes, including block matching and non-local means.
This repo contains the Matlab code package of Self-Convolution which focuses on equivalent implementation of block matching, which includes 2D-patch and 3D-patch versions of Self-Convolution (dimension of the reference image patch). For each version, we provide a demo to show Self-Convolution can speed up the non-local denoising algorithm. To be specific, SAIST as an example method relying on 2D patches, and our proposed multi-modality image denoising method Self-MM as example of 3D patch.
The Self-Convolution functions can be plugged in any block matching based image restoration method, just follow the similar usage steps.
- 2D Patch
Example method: SAIST
Usage:
-
replace
Block_matching.m
function with ourself_convolution_2d.m
function (2d here refers to the two-dimensional search window) -
run
Denoising_Main.m
(a gray-scale image denoising demo)
- 3D Patch
Example method: Self-MM
Usage: run demo_rgbnir_denoising.m
(a RGB-NIR image denoising demo)
Runtime (in seconds) comparisons of non-local algorithms using BM and Self-Convolution, for denoising 512 * 512 single-channel images (first 7 rows) and 256 * 256 * q multi-channel images (last 3 rows), where BMtime% denotes the runtime portion of BM.
Method | Original Runtime | Self-Conv Runtime | BMtime% | Original BM | Self-Conv | Speed-Ups |
---|---|---|---|---|---|---|
SAIST | 708.2 | 562.2 | 32.0% | 227.0 | 78.6 | 3X |
WNNM | 63.2 | 43.8 | 36.9% | 23.3 | 7.8 | 3X |
STROLLR | 87.7 | 68.9 | 36.7% | 38.2 | 13.3 | 3X |
GHP | 412.6 | 218.3 | 69.9% | 288.6 | 94.2 | 3X |
NCSR | 134.7 | 82.4 | 57.1% | 76.9 | 28.1 | 3X |
PGPD | 305.2 | 89.6 | 85.3% | 260.3 | 41.3 | 6X |
RRC | 601.2 | 505.6 | 26.9% | 161.8 | 74.2 | 2X |
MCWNNM | 2899.0 | 2371.3 | 15.8% | 458.6 | 61.6 | 8X |
SALT | 375.9 | 113.8 | 75.4% | 294.8 | 33.2 | 9X |
Self-MM | 139.0 | 44.3 | 78.8% | 109.5 | 16.3 | 7X |
All the experiments are carried out in the Matlab (R2019b) environmentrunning on a PC with Intel(R) Core(TM) i9-10920K CPU 3.50GHz.
Paper available here. Long Journal Version preprint.
In case of use, please cite our publication:
L. Guo, Z. Zha, S. Ravishankar and B. Wen, "Self-Convolution: A Highly-Efficient Operator for Non-Local Image Restoration," ICASSP 2021.
Bibtex:
@INPROCEEDINGS{9414124,
author={Guo, Lanqing and Zha, Zhiyuan and Ravishankar, Saiprasad and Wen, Bihan},
booktitle={ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
title={Self-Convolution: A Highly-Efficient Operator for Non-Local Image Restoration},
year={2021},
volume={},
number={},
pages={1860-1864},
doi={10.1109/ICASSP39728.2021.9414124}}
@article{guo2022exploiting,
title={Exploiting Non-Local Priors via Self-Convolution For Highly-Efficient Image Restoration},
author={Guo, Lanqing and Zha, Zhiyuan and Ravishankar, Saiprasad and Wen, Bihan},
journal={IEEE Transactions on Image Processing},
year={2022},
publisher={IEEE}
}