/Awesome-Face-Restoration

A comprehensive summary of deep face restoration methods.

Survey on Deep Face Restoration: From Non-blind to Blind and Beyond

A comprehensive statistic on methods related to deep face restoration.
[Paper]   [Supplementary Material]

| | | | |

| | | | |

💥 Note: More visual comparisons can be found in the Paper and Supplementary Material.

📋 Citation

@article{li2023survey,
  title={Survey on Deep Face Restoration: From Non-blind to Blind and Beyond},
  author={Li, Wenjie and Wang, Mei and Zhang, Kai and Li, Juncheng and Li, Xiaoming and Zhang, Yuhang and Gao, Guangwei and Deng, Weihong and Lin, Chia-Wen},
  journal={arXiv preprint arXiv:2309.15490},
  year={2023}
}

Table of contents

Non-Blind Tasks

Pub
 Paper
 Technology
AAAI2015 Learning Face Hallucination in the Wild CNN-based
ECCV2016 Deep cascaded bi-network for face hallucination CNN-based
ECCV2016 Ultra-resolving face images by discriminative generative networks GAN-based
ICCV2017 Wavelet-SRNet: A Wavelet-based CNN for Multi-scale Face Super Resolution Wavelet transform
CVPR2017 Attention-Aware Face Hallucination via Deep Reinforcement Learning Attention-based
CVPR2017 Hallucinating very low-resolution unaligned and noisy face images by transformative discriminative autoencoders GAN-based
IJCAI2017 Learning to Hallucinate Face Images via Component Generation and Enhancement Prior-based
ICASSP2017 Face Hallucination Based on Key Parts Enhancement CNN-based
CVPR2018 FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors Prior-based
CVPRW2018 Attribute Augmented Convolutional Neural Network for Face Hallucination GAN-based
ECCV2018 Super-Identity Convolutional Neural Network for Face Hallucination Prior-based
ECCV2018 Face Super-resolution Guided by Facial Component Heatmaps Prior-based
IJCAI2018 Deep CNN Denoiser and Multi-layer Neighbor Component Embedding for Face Hallucination CNN-based
ICASSP2018 FACE HALLUCINATION BASED ON KEY PARTS ENHANCEMENT Prior-based
CVPRW2019 Exemplar Guided Face Image Super-Resolution without Facial Landmarks Prior/GAN-based
BMVC2019 Progressive Face Super-Resolution via Attention to Facial Landmark Prior/GAN-based
AAAI2019 Residual Attribute Attention Network for Face Image Super-Resolution Attention-based
TMM2019 ATMFN: Adaptive-threshold-based Multi-model Fusion Network for Compressed Face Hallucination CNN/GAN/RNN-based
IJCV2019 Joint Face Hallucination and Deblurring via Structure Generation and Detail Enhancement Prior-based
PR2019 Face hallucination from low quality images using definition-scalable inference Frequency-based
TIP2020 Learning Spatial Attention for Face Super-Resolution Attention-based
TIP2020 Deblurring Face Images using Uncertainty Guided Multi-Stream Semantic Networks Prior-based
TMM2020 Learning Face Image Super-Resolution through Facial Semantic Attribute Transformation and Self-Attentive Structure Enhancement Prior-based
TNNLS2020 Dual-Path Deep Fusion Network for Face Image Hallucination CNN-based
ECCV2020 Face Super-Resolution Guided by 3D Facial Priors Prior-based
CVPR2020 Deep Face Super-Resolution with Iterative Collaboration between Attentive Recovery and Landmark Estimation Prior-based
CVPR2020 MSG-GAN: Multi-Scale Gradients for Generative Adversarial Networks GAN-based
AAAI2020 Facial Attribute Capsules for Noise Face Super Resolution Prior-based
MM2020 PCA-SRGAN: Incremental Orthogonal Projection Discrimination for Face Super-resolution GAN-based
ICASSP2020 PARSING MAP GUIDED MULTI-SCALE ATTENTION NETWORK FOR FACE HALLUCINATION Prior-based
WACV2020 Component Attention Guided Face Super-Resolution Network: CAGFace Prior-based
FG2020 ATFaceGAN: Single Face Image Restoration and Recognition from Atmospheric Turbulence GAN-based
TBIOM2021 Semantic-Driven Face Hallucination Based on Residual Network Prior-based
TBBLS2021 E-ComSupResNet: Enhanced Face Super-Resolution Through Compact Network CNN-based
TCYB2021 Sequential Gating Ensemble Network for Noise Robust Multiscale Face Restoration CNN-based
TIP2021 Features Guided Face Super-Resolution via Hybrid Model of Deep Learning and Random Forests Prior-based
TPAMI2021 Face restoration via plug-and-play 3d facial priors Prior-based
CVPR2021 GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolutio GAN/Prior-based
MM2021 Face Hallucination via Split-Attention in Split-Attention Network Attention-based
ICME2021 HEATMAP-AWARE PYRAMID FACE HALLUCINATION Prior-based
ICME2021 SAAN: SEMANTIC ATTENTION ADAPTATION NETWORK FOR FACE SUPER-RESOLUTION Prior-based
ICME2021 ORGAN-BRANCHED CNN FOR ROBUST FACE SUPER-RESOLUTION GAN/Prior-based
ICME2021 FACE SUPER-RESOLUTION THROUGH DUAL-IDENTITY CONSTRAINT Prior-based
ICIP2021 Progressive Face Super-Resolution with Non-Parametric Facial Prior Enhancement Prior-based
TCSVT2022 Progressive Face Super-Resolution with Non-Parametric Facial Prior Enhancement Attention-based
TCSVT2022 Propagating Facial Prior Knowledge for Multitask Learning in Face Super-Resolution Prior-based
TNNLS2022 Rethinking Prior-Guided Face Super-Resolution: A New Paradigm With Facial Component Prior Prior-based
TIP2022 Attention-Driven Graph Neural Network for Deep Face Super-Resolution Attention-based
TPAMI2022 Image Super-Resolution via Iterative Refinement Diffusion-based
Nercom2022 SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models Diffusion-based
ICME2022 RCNet: Recurrent Collaboration Network Guided by Facial Priors for Face Super-Resolution Prior-based
IJCB2022 Face Super-Resolution with Progressive Embedding of Multi-scale Face Priors Prior-based
ICME2022 RCNet: Recurrent Collaboration Network Guided by Facial Priors for Face Super-Resolution Prior-based
Arxiv2022 Bi-Noising Diffusion: Towards Conditional Diffusion Models with Generative Restoration Priors Diffusion-based
Arxiv2022 Stochastic Attribute Modeling for Face Super-Resolution Prior-based
Arxiv2022 HAZE-Net: High-Frequency Attentive Super-Resolved Gaze Estimation in Low-Resolution Face Images Frequency-based
TMM2023 An Efficient Latent Style Guided Transformer-CNN Framework for Face Super-Resolution Attention-based
TMM2023 SCTANet: A Spatial Attention-Guided CNN-Transformer Aggregation Network for Deep Face Image Super-Resolution Attention-based
TMM2023 Exploiting Multi-scale Parallel Self-attention and Local Variation via Dual-branch Transformer-CNN Structure for Face Super-resolution Attention-based
TBIOM2023 Super-Resolving Face Image by Facial Parsing Information Prior-based
TIP2023 CTCNet: A CNN-Transformer Cooperation Network for Face Image Super-Resolution Attention-based
TIM2023 Deep HyFeat Based Attention in Attention Model for Face Super-Resolution GAN/Attention-based
PR2023 A Composite Network Model for Face Super-Resolution with Multi-Order Head Attention Facial Priors Attention/Prior-based
CVPR2023 Spatial-Frequency Mutual Learning for Face Super-Resolution Attention-based
CVPR2023 Implicit Diffusion Models for Continuous Super-Resolution Diffusion-based
IJCAI2023 Learning Attention from Attention: Effcient Self-Refnement Transformer for Face Super-Resolution Attention-based

Blind Tasks

Pub
 Paper
 Technology
CVPRW2017 Deep Face Deblurring CNN-based
ECCV2018 Learning to Super-Resolve Blurry Face and Text Images GAN-based
ECCV2018 Learning Warped Guidance for Blind Face Restoration CNN-based
ECCV2018 Attribute-guided face generation using conditional cyclegan CycleGAN-based
ECCV2018 To learn image super-resolution, use a GAN to learn how to do image degradation first CycleGAN-based
CVPR2018 Super-fan: Integrated facial landmark localization and super-resolution of real-world low resolution faces in arbitrary poses with gans Prior/GAN-based
CVPR2018 Exploiting Semantics for Face Image Deblurring Prior-based
CVPRW2018 Learning Face Deblurring Fast and Wide Prior-based
CVPRW2018 Learning Face Deblurring Fast and Wide CNN-based
ICCV2019 DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better GAN-based
ICIP2019 GUIDED CYCLEGAN VIA SEMI-DUAL OPTIMAL TRANSPORT FOR PHOTO-REALISTIC FACE SUPER-RESOLUTION CycleGAN-based
IJCV2020 Exploiting Semantics for Face Image Deblurring Prior-based
IJCV2020 Identity-preserving Face Recovery from Stylized Portraits Prior-based
AAAI2020 Learning to Deblur Face Images via Sketch Synthesis GAN-based
ECCV2020 Blind Face Restoration via Deep Multi-scale Component Dictionaries GAN/Prior-based
CVPR2020 Enhanced Blind Face Restoration with Multi-Exemplar Images and Adaptive Spatial Feature Fusion GAN/Prior-based
CVPR2020 Image Processing Using Multi-Code GAN Prior GAN/Prior-based
MM2020 HiFaceGAN: Face Renovation via Collaborative Suppression and Replenishment GAN-based
Arxiv2020 DeepDeblur: Fast one-step blurry face images restoration GAN-based
CVPR2021 Progressive Semantic-Aware Style Transformation for Blind Face Restoration GAN-based
CVPR2021 Progressive Semantic-Aware Style Transformation for Blind Face Restoration GAN-based
CVPR2021 Progressive Semantic-Aware Style Transformation for Blind Face Restoration GAN/Prior-based
CVPR2021 Towards Real-World Blind Face Restoration with Generative Facial Prior GAN/Prior-based
CVPR2021 GAN Prior Embedded Network for Blind Face Restoration in the Wild GAN/Prior-based
Arxiv2021 Universal Face Restoration With Memorized Modulation GAN-based
Arxiv2021 Implicit Subspace Prior Learning for Dual-Blind Face Restoration GAN-based
CVPR2022 RestoreFormer: High-Quality Blind Face Restoration
from Undegraded Key-Value Pairs		 GAN/Attention/Prior-based
CVPR2022 Blind Face Restoration via Integrating Face Shape and Generative Priors GAN/Prior-based
CVPR2022 Rethinking Deep Face Restoration GAN/Prior-based
CVPR2022 GCFSR: a Generative and Controllable Face Super Resolution Method Without Facial and GAN Priors GAN-based
CVPRW2022 Unpaired Face Restoration via Learnable Cross-Quality Shift CycleGAN-based
AAAI2022 Panini-Net: GAN Prior Based Degradation-Aware Feature Interpolation for Face Restoration GAN/Prior-based
ECCV2022 VQFR: Blind Face Restoration with Vector-Quantized Dictionary and Parallel Decoder GAN/Prior-based
ECCV2022 From Face to Natural Image: Learning Real Degradation for Blind Image Super-Resolution GAN-based
Nips2022 Towards Robust Blind Face Restoration with Codebook Lookup Transformer GAN/Prior-based
ICME2022 FEATURE-GUIDED BLIND FACE RESTORATION WITH GAN PRIOR GAN/Prior-based
WACV2022 Deep Feature Prior Guided Face Deblurring GAN/Prior-based
TCSVT2022 DEAR-GAN: Degradation-Aware Face Restoration with GAN Prior GAN/Prior-based
TPAMI2022 GLEAN: Generative Latent Bank for Image Super-Resolution and Beyond GAN/Prior-based
TPAMI2022 Learning Dual Memory Dictionaries for Blind Face Restoration GAN/Prior-based
SIGGRAPH2022 Face Deblurring using Dual Camera Fusion on Mobile Phones CNN-based
Arxiv2022 FaceFormer: Scale-aware Blind Face Restoration with Transformers GAN/Attention-based
Arxiv2022 Multi-prior learning via neural architecture search for blind face restoration GAN/Prior-based
Arxiv2022 Blind Face Restoration: Benchmark Datasets and a Baseline Model GAN/Attention-based
Arxiv2022 Enhancing Quality of Pose-varied Face Restoration with Local Weak Feature Sensing and GAN Prior GAN/Prior-based
Arxiv2022 Difface: Blind Face Restoration with Diffused Error Contraction Diffusion-based
Arxiv2022 Face Deblurring Based on Separable Normalization and Adaptive Denormalization Prior/GAN-based
CVPR2023 DR2: Diffusion-based Robust Degradation Remover for Blind Face Restoration Diffusion-based
CVPRW2023 TFRGAN: Leveraging Text Information for Blind Face Restoration with Extreme Degradation GAN/Prior-based
CVPRW2023 FRR-Net: A Real-Time Blind Face Restoration and Relighting Network GAN/Prior-based
AAAI2023 GAN Prior Based Null-Space Learning for Consistent Super-resolution GAN-based
IJCAI2023 Analyzing and Combating Attribute Bias for Face Restoration GAN/Prior-based
ICML2023 ZERO-SHOT IMAGE RESTORATION USING DENOISING DIFFUSION NULL-SPACE MODEL Diffusion-based
ICCV2023 Towards Authentic Face Restoration with Iterative Diffusion Models and Beyond Diffusion-based
MM2023 DiffBFR: Bootstrapping Diffusion Model Towards Blind Face Restoration Diffusion-based
Nips2023 PGDiff: Guiding Diffusion Models for Versatile Face Restoration via Partial Guidance Diffusion-based
TIP2023 Semi-Cycled Generative Adversarial Networks for Real-World Face Super-Resolution> CycleGAN-based
TPAMI2023 Old Photo Restoration via Deep Latent Space Translation GAN/Prior-based
TPAMI2023 Multi-Modality Deep Restoration of Extremely Compressed Face Videos GAN/Prior-based
TPAMI2023 RestoreFormer++: Towards Real-World Blind Face Restoration from Undegraded Key-Value Pairs Prior/GAN-based
TCSVT2023 Blind Face Restoration for Under-Display Camera via Dictionary Guided Transformer Prior/Attention-based
WACV2023 AT-DDPM: Restoring Faces degraded by Atmospheric Turbulence using Denoising Diffusion Probabilistic Models Diffusion-based
Arxiv2023 ResDiff: Combining CNN and Diffusion Model for Image Super-Resolution Diffusion-based
Arxiv2023 DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior Diffusion-based
Arxiv2023 Exploiting Diffusion Prior for Real-World Image Super-Resolution Diffusion-based
Arxiv2023 HDTR-Net: A Real-Time High-Definition Teeth Restoration Network for Arbitrary Talking Face Generation Methods CNN-based
Arxiv2023 Towards Real-World Blind Face Restoration with Generative Diffusion Prior Prior/Diffusion-based
ICLR2024 DUAL ASSOCIATED ENCODER FOR FACE RESTORATION Prior/GAN-based
Arxiv2024 WaveFace: Authentic Face Restoration with Efficient Frequency Recovery Diffusion-based
Arxiv2024 DiffMAC: Diffusion Manifold Hallucination Correction for High Generalization Blind Face Restoration Diffusion-based
Arxiv2024 PFStorer: Personalized Face Restoration and Super-Resolution Diffusion-based

Joint Tasks

Joint Face Completion

Pub
 Paper
 Technology
ICME2019 Facial Image Inpainting Using Multi-level Generative Network CNN-based
FG2020 IF-GAN: Generative Adversarial Network for Identity Preserving Facial Image Inpainting and Frontalization GAN-based
TBOIM2020 FCSR-GAN: Joint Face Completion and Super-Resolution via Multi-Task Learning GAN/Prior-based
TCSVT2020 Occluded Face Recognition in the Wild by Identity-Diversity Inpainting GAN/Prior-based
ICIP2022 Swin-CasUNet: Cascaded U-Net with Swin Transformer for Masked Face Restoration Attention-based
WACV2022 3DFaceFill: An Analysis-By-Synthesis Approach to Face Completion Prior/GAN-based
TMM2022 Pluralistic Face Inpainting with Transformation of Attribute Information GAN/Prior-based
TMM2023 FT-TDR: Frequency-Guided Transformer and Top-Down Refinement Network for Blind Face Inpainting GAN/Prior-based
TMM2023 JDSR-GAN: Constructing an Efficient Joint Learning Network for Masked Face Super-Resolution GAN/Prior-based
TCSVT2022 SwapInpaint: Identity-Specific Face Inpainting With Identity Swapping GAN/Prior-based
JVCI2023 Joint Face Completion and Super-resolution using Multi-scale Feature Relation Learning GAN/Prior-based
ICASSP2023 Fine-Grained Blind Face Inpainting with 3D Face Component Disentanglement Prior-based
TIP2023 Deep Face Video Inpainting via UV Mapping Attention/GAN/Prior-based
PR2023 Joint face completion and super-resolution using multi-scale feature relation learning Attention/GAN/Prior-based

Joint Face Frontalization

Pub
 Paper
 Technology
TPAMI2019 Can We See More? Joint Frontalization and Hallucination of Unaligned Tiny Faces GAN-based
FG2020 IF-GAN: Generative Adversarial Network for Identity Preserving Facial Image Inpainting and Frontalization GAN-based
TCSVT2021 Joint Face Image Restoration and Frontalization for Recognition GAN/Prior-based
TCSVT2021 Simultaneous Face Completion and Frontalization via Mask Guided Two-Stage GAN GAN/Attention-based
TIP2021 Face Hallucination With Finishing Touches GAN/Prior-based

Joint Face Alignment

Pub
 Paper
 Technology
AAAI2017 Face Hallucination with Tiny Unaligned Images by Transformative Discriminative Neural Networks GAN-based
CVPR2017 Hallucinating Very Low-Resolution Unaligned and Noisy Face Images by Transformative Discriminative Autoencoders∗ GAN-based
CVPR2018 Semantic Face Hallucination: Super-Resolving Very Low-Resolution Face Images with Supplementary Attributes GAN-based
AAAI2020 Joint Super-Resolution and Alignment of Tiny Faces Prior-based
IJCV2020 Hallucinating Unaligned Face Images by Multiscale Transformative Discriminative Networks GAN-based

Joint Face Recogntion

Pub
 Paper
 Technology
ECCV2018 Super-Identity Convolutional Neural Network for Face Hallucination Prior-based
ICIP2019 Low-Resolution Face Recognition Based on Identity-Preserved Face Hallucination CNN-based
TIP2019 SiGAN: Siamese Generative Adversarial Network for Identity-Preserving Face Hallucination GAN/Prior-based
TIP2019 Face Hallucination Using Cascaded Super-Resolution and Identity Priors GAN/Prior-based
Nercom2019 Edge and Identity Preserving Network for Face Super-Resolution GAN/Prior-based
Arxiv2019 Verification of Very Low-Resolution Faces Using An Identity-Preserving Deep Face Super-Resolution Network GAN/Prior-based
IJCV2019 Wavelet Domain Generative Adversarial Network for Multi-scale Face Hallucination GAN/Prior-based
TPAMI2020 Adversarial Cross-Spectral Face Completion for NIR-VIS Face Recognition GAN-based

Joint Face Illumination Compensation

Pub
 Paper
 Technology
ICB2019 SeLENet: A Semi-Supervised Low Light Face Enhancement Method for Mobile Face Unlock CNN-based
CVPR2020 Copy and Paste GAN: Face Hallucination from Shaded Thumbnails CNN-based
CVPR2020 From Less to More: Spectral Splitting and Aggregation Network for Hyperspectral Face Super-Resolution CNN-based
ICME2020 Learning To See Faces In The Dark CNN-based
Arxiv2021 Network Architecture Search for Face Enhancement CNN-based
TPAMI2021 Recursive Copy and Paste GAN: Face Hallucination From Shaded Thumbnails GAN-based
TMM2023 Learning to Hallucinate Face in the Dark CNN-based
AAAI24 Low-Light Face Super-resolution via Illumination, Structure, and Texture Associated Representation Attention-based

Joint Face Fairness

Pub
 Paper
 Technology
ICIP2021 Identity and Attribute Preserving Thumbnail Upscaling Prior-based
ICML2021 Fairness for image generation with uncertain sensitive attributes CNN-based
TPAMI2022 EDFace-Celeb-1M: Benchmarking Face Hallucination With a Million-Scale Dataset Dataset
Arxiv2022 Blind Face Restoration: Benchmark Datasets and a Baseline Model Dataset

Joint Face 3D Face Reconstruction

Pub
 Paper
 Technology
FG202O Face Denoising and 3D Reconstruction from A Single Depth Image CNN-based
IJCB2021 3D Face Point Cloud Super-Resolution Network CNN-based
Sensors2022 Incomplete Region Estimation and Restoration of 3D Point Cloud Human Face Datasets CNN-based
CVPR2022 Learning to Restore 3D Face from In-the-Wild Degraded Images Prior/GAN-based

Perfermance

Non-Blind Face Super-Resoution

Blind Face Restoration

Blind Face Super-Resoution

Joint Face Restoration

💥 Note: It can be found in the Supplementary Material.

Benchmarks

Populur Datasets

Dataset Quantity Introduction Year
FFHQ 70,000 non-paired dataset for training 2018
CelebA 202,599 non-paired dataset for training 2015
CelebA-HQ 30,000 non-paired dataset for training 2020
LFW 13,233 non-paired dataset for training 2008
Multi-PIE 75,000 non-paired dataset for training 2010
Helen 2,330 non-paired dataset for testing 2012
CelebA-Test 3,000 non-paired dataset for testing 2021
CelebChild-Test 180 non-paired real-world dataset for testing 2021
CelebAdult-Test 180 non-paired real-world dataset for testing 2021
WebPhoto-Test 407 non-paired real-world dataset for testing 2021
LFW-Test 1,711 non-paired real-world dataset for testing 2021
Wider-Test 970 non-paired real-world dataset for testing 2022

💥 Note: More datasets can be found in the Paper.

Evaluation Metrics

🚩 Note: A detailed description of the evaluation metrics and how to use it can be found here.

📧 Contact

If you have any question, please email lewj2408@gmail.com