Awesome-diffusion-model-for-image-processing: A repository from Aliengmx

Purpose: We aim to provide a summary of diffusion model-based image processing, including restoration, enhancement, coding, and quality assessment. More papers will be summarized.

Xin Li, Yulin Ren, Xin Jin, Cuiling Lan, Xingrui Wang, Wenjun Zeng, Xinchao Wang, Zhibo Chen

University of Science and Technology of China (USTC), National University of Singapore (NUS), Microsoft Research Asia (MSRA), Eastern Institute of Technology (EIT)

Brief intro: The survey for diffusion model-based IR has been released.

📌 About new works. If you want to incorporate your studies (e.g., the link of paper or project) on diffusion model-based image processing in this repository. Welcome to raise an issue or email us. We will incorporate it into this repository and our survey report as soon as possible.

🌟 Features

Survey for diffusion model-based Image Restoration (Arxiv version is released)
Summary for diffusion model-based Image/Video Compression
Summary for diffusion model-based Quality Assessment

Diffusion model-based Image Restoration/Enhancement

Diffusion model for Image Super resolution
Diffusion model for Image Deblurring
Diffusion model for Image Inpainting
Diffusion model for Image Shadow Removal
Diffusion model for Image Restoration
Diffusion model for Image Denoising
Diffusion model for Medical IR
Diffusion model for other tasks
Benchmark Datasets

Image Super-Resolution

Models	Paper	First Author	Training Way	Venue	Topic	Project
SR3	Image super-resolution via iterative refinement	Chitwan Saharia	Supervised	TPAMI2022	Super-resolution	Github
SRDiff	SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models	Haoying Li	Supervised	Neurocomputing 2022	Super-resolution	Github
CDPMSR	CDPMSR: Conditional Diffusion Probabilistic Models for Single Image Super-Resolution	Axi Niu	Supervised	Preprint'23	Super-resolution
ResDiff	ResDiff: Combining cnn and diffusion model for image superresolution	Shuyao Shang	Supervised	Preprint'23	Super-resolution
SDE-SR	Face super-resolution using stochastic differential equations	Marcelo dos Santos	Supervised	SIBGRAPI 2022	Super-resolution	Github
IDM	Implicit diffusion models for continuous super-resolution	Sicheng Gao	Supervised	CVPR2023	Super-resolution	Github
DiffIR	DiffIR: Efficient diffusion model for image restoration	Bin Xia	Supervised	Preprint'23	Super-resolution
CDM	Cascaded Diffusion Models for High Fidelity Image Generation	Jonathan Ho	Supervised	J.Mach.Learn.Res.2022	Super-resolution
StableSR	Exploiting diffusion prior for real-world image super-resolution	Jianyi Wang	Supervised	Preprint'23	Real-World Super-resolution	Github
SR3+	Denoising diffusion probabilistic models for robust image super-resolution in the wild	Hshmat Sahak	Supervised	Preprint'23	(Real-World) Super-resolution
ResShift	ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting	Zongsheng Yue	Supervised	Preprint'23	Real-World Super-resolution	Github
PartDiff	PartDiff: Image Super-resolution with Partial Diffusion Models	Kai Zhao	Supervised	Preprint'23	Super-resolution
Diwa	Waving Goodbye to Low-Res: A Diffusion-Wavelet Approach for Image Super-Resolution	Brian Moser	Supervised	Preprint'23	Super-resolution
YODA	YODA: You Only Diffuse Areas. An Area-Masked Diffusion Approach For Image Super-Resolution	Brian B. Moser	Supervised	Preprint'23	Super-resolution

Image Deblurring

Paper	First Author	Training Way	Venue	Topic	Project
Deblurring via stochastic refinement	Jay Whang	Supervised	CVPR2022	Image Deblurring
Score-Based Diffusion Models as Principled Priors for Inverse Imaging	Berthy T. Feng	Zero-shot	PrePrint'23	Image Denoising, Image Deblurring
BlindDPS:Parallel diffusion models of operator and image for blind inverse problems	Hyungjin Chung	Zero-shot	CVPR2023	Blind Deblurring	Github

Image Inpainting

Model	Paper	First Author	Training Way	Venue	Topic	Project
RePaint	RePaint: Inpainting using denoising diffusion probabilistic models	Andreas Lugmayr	Zero-shot	CVPR2022	Image Inpainting	Github
CoPaint	Towards coherent image inpainting using denoising diffusion implicit models	Guanhua Zhang	Zero-shot	PrePrint'23	Image Inpainting	Github

Image Shadow Removal

Model	Paper	First Author	Training Way	Venue	Topic	Project
DeS3	DeS3: Attention-driven Self and Soft Shadow Removal using ViT Similarity and Color Convergence	Yeying Jin	Supervised	Preprint'23	Image Shadow Removal
ShadowDiffusion	Shadowdiffusion: When degradation prior meets diffusion model for shadow removal	Lanqing Guo	Supervised	CVPR2023	Image Shadow Removal	Github

Image Restoration

Model	Paper	First Author	Training Way	Venue	Topic	Project
Palette	Palette: Image-toimage diffusion models	Chitwan Saharia	Supervised	SIGGRAPH 2022	Multi-task Restoration	Github
Refusion	Refusion: Enabling large-size realistic image restoration with latent-space diffusion models	Ziwei Luo	Supervised	CVPRW 2023	Multi-task Restoration
InDI	Inversion by direct iteration: An alternative to denoising diffusion for image restoration	Mauricio Delbracio	Supervised	Preprint'22	Multi-task Restoration
Weather-Diff	Restoring vision in adverse weather conditions with patch-based denoising diffusion models	Ozan Özdenizci	Supervised	TPAMI2022	Multi-task Restoration	Github
IR-SDE	Image Restoration with Mean-Reverting Stochastic Differential Equations	Ziwei Luo	Supervised	ICML 2023	Multi-task Restoration	Github
SUD^2	SUD ˆ2: Supervision by denoising diffusion models for image reconstruction	Matthew A. Chan	Supervised	Preprint'23	Multi-task Restoration
ILVR	ILVR: Conditioning method for denoising diffusion probabilistic models	Jooyoung Choi	Zero-shot	ICCV2021	Image Restoration	Github
CCDF	Come-closer-diffusefaster: Accelerating conditional diffusion models for inverse problems through stochastic contraction	Hyungjin Chung	Zero-shot	CVPR2022	Multi-task Restoration
SNIPS	SNIPS: Solving noisy inverse problems stochastically	Bahjat Kawa	Zero-shot	NeurIPS 2021	Multi-task Restoration	Github
DDRM	Denoising diffusion restoration models	Bahjat Kawar	Zero-shot	ICLR2022	Multi-task Restoration	Github
DDNM	Zero-shot image restoration using denoising diffusion null-space model	Yinhuai Wang	Zero-shot	ICLR2023	Multi-task Restoration	Github
DPS	Diffusion Posterior Sampling for General Noisy Inverse Problems	Hyungjin Chung	Zero-shot	ICLR 2023	Multi-task Restoration	Github
DiracDiffusion	DiracDiffusion: Denoising and incremental reconstruction with assured data-consistency	Zalan Fabian	Zero-shot	NeurIPS 2022	Multi-task Restoration
IIGDM	Pseudoinverse-guided diffusion models for inverse problems	Jiaming Song	Zero-shot	ICLR 2023	Multi-task Restoration
GDP	Generative diffusion prior for unified image restoration and enhancement	Ben Fei	Zero-shot	CVPR2023	Multi-task Restoration	Github
DiffPIR	Denoising diffusion models for plug-and-play image restoration	Yuanzhi Zhu	Zero-shot	CVPR 2023	Multi-task Restoration	Github
RED-Diff	A variational perspective on solving inverse problems with diffusion models	Morteza Mardani	Zero-shot	PrePrint'22	Multi-task Restoration
--	Multiscale Structure Guided Diffusion for Image Deblurring	Mengwei Ren	Supervised	Preprint'22	Blind Restoration
DriftRec	DriftRec: Adapting diffusion models to blind image restoration tasks	Simon Welker	Supervised	Preprint'22	Blind Restoration
DR2	DR2: Diffusion-based robust degradation remover for blind face restoration	Zhixin Wang	Zero-shot	CVPR2023	Blind Face Restoration
DiffBFR	DiffBFR: Bootstrapping diffusion model towards blind face restoration	Xinmin Qiu	Supervised	Preprint'23	Blind Face Restoration
DifFace	DifFace: Blind face restoration with diffused error contraction	Zongsheng Yu	Zero-shot	PrePrint'23	Blind Restoration	Github
GibbsDDRM	GibbsDDRM: A partially collapsed gibbs sampler for solving blind inverse problems with denoising diffusion restoration	Naoki Murata	Zero-shot	PrePrint'23	Blind Restoration
ADIR	ADIR: Adaptive diffusion for image reconstruction	Shady Abu-Hussein	Zero-shot	PrePrint'22	Blind Restoration
--	Synthesizing realistic image restoration training pairs: A diffusion approach	Tao Yang	Supervised	Preprint'22	Blind Restoration

Image Denosing

Paper	First Author	Training Way	Venue	Topic	Project
Diffusion model for generative image denoising	Yutong Xie	Supervised	Preprint'23	Image Denoising
Score-Based Diffusion Models as Principled Priors for Inverse Imaging	Berthy T. Feng	Zero-shot	PrePrint'23	Image Denoising, Image Deblurring

Medical Restoration (MRI,CT)

Model	Paper	First Author	Training Way	Venue	Topic	Project
MCG	Improving diffusion models for inverse problems using manifold constraints	Hyungjin Chung	Zero-shot	NeurIPS 2022	CT Reconstruction	Github
--	Solving inverse problems in medical imaging with score-based generative models	Yang Song	Zero-shot	ICLR 2022	CT Reconstruction
AdaDiff	Adaptive diffusion priors for accelerated mri reconstruction	Alper Güngör	Supervised	Preprint'23	MRI Reconstruction
HFS-SDE	High-Frequency Space Diffusion Models for Accelerated MRI	Chentao Cao	Supervised	Preprint'22	MRI Reconstruction

Other tasks

Model	Paper	First Author	Training Way	Venue	Topic	Project
--	Jpeg artifact correction using denoising diffusion restoration models	Bahjat Kawar	Zero-shot	NeurIPS 2022	JPEG Artifacts Removal	Github
--	Improved Diffusion-based Image Colorization via Piggybacked Models	Hanyuan Liu	Zero-shot	PrePrint'23	Image Colorization	Github
PyDiff	Pyramid Diffusion Models For Low-light Image Enhancement	Dewei Zhou	Supervised	Preprint'23	Low-light Enhancemnent	Github
DiffLL	Lowlight image enhancement with wavelet-based diffusion models	Hai Jiang	Supervised	Preprint'23	Low-light Enhancemnent
RainDiffusion	RainDiffusion: When Unsupervised Learning Meets Diffusion Models for Real-world Image Deraining	Mingqiang Wei	Supervised	Preprint'23	Image Deraining
DiffGAR	DiffGAR: Model-agnostic restoration from generative artifacts using image-to-image diffusion models	Yueqin Yin	Supervised	Preprint'22	Generative Artifacts Removal

Benchmark Datasets

Dataset	Task	Usage	Year
DIV2K	Image Super-resolution	Training,Testing	2017
Flickr2K	Image Super-resolution	Training	2017
Set5	Image Super-resolution	Testing	2012
Set14	Image Super-resolution	Testing	2012
BSD100	Image Super-resolution	Testing	2012
Manga109	Image Super-resolution	Testing	2015
Urban100	Image Super-resolution	Testing	2015
OST300	Image Super-resolution	Testing	2018
DIV8K	Image Super-resolution	Training,Testing	2019
RealSR	Image Super-resolution	Training,Testing	2019
DRealSR	Image Super-resolution	Training,Testing	2020
GoPro	Image Deblurring	Training,Testing	2017
HIDE	Image Deblurring	Training,Testing	2019
RealBlur	Image Deblurring	Training,Testing	2020
Kodak	Image Denoising	Testing	1999
CBSD68	Image Denoising	Testing	2001
McMaster	Image Denoising	Testing	2011
ImageNet	Image Classification	Training,Testing	2010
ImageNet1k	Image Classification	Testing	2020
LSUN	Image Classification	Training,Testing	2015
Places365	Image Classification	Training,Testing	2019
LFW	Face Generation	Training	2008
FFHQ	Face Generation	Training	2019
Celeba-HQ	Face Generation	Training	2018
AFHQ	Face Generation	Training	2020
CelebA	Face Generation	Training	2015
ISTD	Image Shadow Removal	Training,Testing	2018
SRD	Image Shadow Removal	Training,Testing	2017
CSD	Image Desnowing	Training,Testing	2021
Snow100k	Image Desnowing	Training,Testing	2017
SRRS	Image Desnowing	Training,Testing	2020
RainDrop	Image Deraining	Training	2018
Outdoor-Rain	Image Deraining	Training,Testing	2019
DDN-data	Image Deraining	Training,Testing	2017
SPA-data	Image Deraining	Training,Testing	2019
Rain100H	Image Deraining	Training,Testing	2017
Rain100L	Image Deraining	Training,Testing	2017
Haze-4K	Image Dehazing	Training	2021
Dense-Haze	Image Dehazing	Training	2019
RESIDE	Image Dehazing	Training	2019

Aliengmx/Awesome-diffusion-model-for-image-processing