A Deep Learning Framework to Reconstruct Face under Mask

This is the official repository for the research work A Deep Learning Framework to Reconstruct Face under Mask. We are updating the available resources in this repo. Stay tuned with us. Thank you.

The pdf version of the full paper: https://arxiv.org/abs/2203.12482

Datasets

Dataset of Mask Segementation and Inpainting: Google Drive || Mendeley

Dataset for Gender Classification: Gender Classified Dataset with Masked Face

Experiments

We have used the Labelme software to create our dataset. You will find the mask segmentation dataset in the above link. Here is one example:

We have perfectly detect the mask as an object. Here is the result you can see:

There is no edges on the face because we have detect the region of the mask properly.

Now, comparison with the ground truth of an image we know.

We have used Adaptive WingLoss for Facial Landmark Calculation.

Trained Models

Inpainting Model

FFHQ

For Male: Generator || Discriminator

For Female: Generator || Discriminator

CelebA

For Male: Generator || Discriminator

For Female: Generator || Discriminator

Mask Segmentation Model:

We have used Mask RCNN to train our dataset. Saved Model Link: Google Drive

Gender Classification Model:

We have used Inception V3 pretrained model and used keras tuner to tune the model for our problem. We trained with the dataset of 89k images separately male and female. Saved Model Link: Google Drive

Citation

If you find our work helpful for your research, please cite our paper.

@INPROCEEDINGS{9736350,  author={Modak, Gourango and Das, Shuvra Smaran and Islam Miraj, Md. Ajharul and Morol, Md. Kishor},  
booktitle={2022 7th International Conference on Data Science and Machine Learning Applications (CDMA)},   
title={A Deep Learning Framework to Reconstruct Face under Mask},   
year={2022},  volume={},  number={},  pages={200-205},  
doi={10.1109/CDMA54072.2022.00038}}

iamshuvra/A-Deep-Learning-Framework-to-Reconstruct-Face-under-Mask