/DeepFAS

🔥Deep Learning for Face Anti-Spoofing

👏 Survey of Deep Face Anti-spoofing 🔥

This is the official repository of "Deep Learning for Face Anti-Spoofing: A Survey", a comprehensive survey of recent progress in deep learning methods for face anti-spoofing (FAS) as well as the datasets and protocols.

Introduction

We present a comprehensive review of recent deep learning methods for face anti-spoofing (mostly from 2018 to 2021). It covers hybrid (handcrafted+deep), pure deep learning, and generalized learning based methods for monocular RGB face anti-spoofing. It also includes multi-modal learning based methods as well as specialized sensor based FAS. It also presents detailed comparision among publicly available datasets, together with several classical evaluation protocols.

🔔 We will update this page frequently~ 🎉🎉🎉


Contents


image


Dataset Year #Live/Spoof #Sub. Setup Attack Types
NUAA 2010 5105/7509(I) 15 N/R Print(flat, wrapped)
YALE Recaptured 2011 640/1920(I) 10 50cm-distance from 3 LCD minitors Print(flat)
CASIA-MFSD 2012 150/450(V) 50 7 scenarios and 3 image quality Print(flat, wrapped, cut), Replay(tablet)
REPLAY-ATTACK 2012 200/1000(V) 50 Lighting and holding Print(flat), Replay(tablet, phone)
Kose and Dugelay 2013 200/198(I) 20 N/R Mask(hard resin)
MSU-MFSD 2014 70/210(V) 35 Indoor scenario; 2 types of cameras Print(flat), Replay(tablet, phone)
UVAD 2015 808/16268(V) 404 Different lighting, background and places in two sections Replay(monitor)
REPLAY-Mobile 2016 390/640(V) 40 5 lighting conditions Print(flat), Replay(monitor)
HKBU-MARs V2 2016 504/504(V) 12 7 cameras from stationary and mobile devices and 6 lighting settings Mask(hard resin) from Thatsmyface and REAL-f
MSU USSA 2016 1140/9120(I) 1140 Uncontrolled; 2 types of cameras Print(flat), Replay(laptop, tablet, phone)
SMAD 2017 65/65(V) - Color images from online resources Mask(silicone)
OULU-NPU 2017 720/2880(V) 55 Lighting & background in 3 sections Print(flat), Replay(phone)
Rose-Youtu 2018 500/2850(V) 20 5 front-facing phone camera; 5 different illumination conditions Print(flat), Replay(monitor, laptop),Mask(paper, crop-paper)
SiW 2018 1320/3300(V) 165 4 sessions with variations of distance, pose, illumination and expression Print(flat, wrapped), Replay(phone, tablet, monitor)
WFFD 2019 2300/2300(I) 140/145(V) 745 Collected online; super-realistic; removed low-quality faces Waxworks(wax)
SiW-M 2019 660/968(V) 493 Indoor environment with pose, lighting and expression variations Print(flat), Replay, Mask(hard resin, plastic, silicone, paper, Mannequin), Makeup(cosmetics, impersonation, Obfuscation), Partial(glasses, cut paper)
Swax 2020 Total 1812(I) 110(V) 55 Collected online; captured under uncontrolled scenarios Waxworks(wax)
CelebA-Spoof 2020 156384/469153(I) 10177 4 illumination conditions; indoor & outdoor; rich annotations Print(flat, wrapped), Replay(monitor tablet, phone), Mask(paper)
RECOD-Mtablet 2020 450/1800(V) 45 Outdoor environment and low-light & dynamic sessions Print(flat), Replay(monitor)
CASIA-SURF 3DMask 2020 288/864(V) 48 High-quality identity-preserved; 3 decorations and 6 environments Mask(mannequin with 3D print)
HiFiMask 2021 13650/40950(V) 75 three mask decorations; 7 recording devices; 6 lighting conditions; 6 scenes Mask(transparent, plaster, resin)
Dataset Year #Live/Spoof #Sub. M&H Setup Attack Types
3DMAD 2013 170/85(V) 17 VIS, Depth 3 sessions (2 weeks interval) Mask(paper, hard resin)
GUC-LiFFAD 2015 1798/3028(V) 80 Light field Distance of 1.5 constrained conditions Print(Inkjet paper, Laserjet paper), Replay(tablet)
3DFS-DB 2016 260/260(V) 26 VIS, Depth Head movement with rich angles Mask(plastic)
BRSU Skin/Face/Spoof 2016 102/404(I) 137 VIS, SWIR multispectral SWIR with 4 wavebands 935nm, 1060nm, 1300nm and 1550nm Mask(silicon, plastic, resin, latex)
Msspoof 2016 1470/3024(I) 21 VIS, NIR 7 environmental conditions Black&white Print(flat)
MLFP 2017 150/1200(V) 10 VIS, NIR, Thermal Indoor and outdoor with fixed and random backgrounds Mask(latex, paper)
ERPA 2017 Total 86(V) 5 VIS, Depth, NIR, Thermal Subject positioned close (0.3∼0.5m) to the 2 types of cameras Print(flat), Replay(monitor), Mask(resin, silicone)
LF-SAD 2018 328/596(I) 50 Light field Indoor fix background, captured by Lytro ILLUM camera Print(flat, wrapped), Replay(monitor)
CSMAD 2018 104/159(V+I) 14 VIS, Depth, NIR, Thermal 4 lighting conditions Mask(custom silicone)
3DMA 2019 536/384(V) 67 VIS, NIR 48 masks with different ID; 2 illumination & 4 capturing distances Mask(plastics)
CASIA-SURF 2019 3000/18000(V) 1000 VIS, Depth, NIR Background removed; Randomly cut eyes, nose or mouth areas Print(flat, wrapped, cut)
WMCA 2019 347/1332(V) 72 VIS, Depth, NIR, Thermal 6 sessions with different backgrounds and illumination; pulse data for bonafide recordings Print(flat), Replay(tablet), Partial(glasses), Mask(plastic, silicone, and paper, Mannequin)
CeFA 2020 6300/27900(V) 1607 VIS, Depth, NIR 3 ethnicities; outdoor & indoor; decoration with wig and glasses Print(flat, wrapped), Replay, Mask(3D print, silica gel)
HQ-WMCA 2020 555/2349(V) 51 VIS, Depth, NIR, SWIR, Thermal Indoor; 14 ‘modalities’, including 4 NIR and 7 SWIR wavelengths; masks and mannequins were heated up to reach body temperature Laser or inkjet Print(flat), Replay(tablet, phone), Mask(plastic, silicon, paper, mannequin), Makeup, Partial(glasses, wigs, tatoo)
PADISI-Face 2021 1105/924(V) 360 VIS, Depth, NIR, SWIR, Thermal Indoor, fixed background, 60-frame sequence of 1984 × 1264 pixel images print(flat), replay(tablet, phone), mask(plastic, silicon, transparent, Mannequin), makeup/tatoo, partial(glasses,funny eye)

  • temp
Method Year Backbone Loss Input Static/Dynamic
DPCNN 2016 VGG-Face Trained with SVM RGB S
Multi-cues+NN 2016 MLP Binary CE loss RGB+OFM D
CNN LBP-TOP 2017 5-layer CNN Binary CE loss, SVM RGB D
DF-MSLBP 2018 Deep forest Binary CE loss HSV+YCbCr S
SPMT+SSD 2018 VGG16 Binary CE loss, SVM, bbox regression RGB, Landmarks S
CHIF 2019 VGG-Face Trained with SVM RGB S
DeepLBP 2019 VGG-Face Binary CE loss, SVM RGB, HSV, YCbCr S
CNN+LBP+WLD 2019 CaffeNet Binary CE loss RGB S
Intrinsic 2019 1D-CNN Trained with SVM Reflection D
FARCNN 2019 Multi-scale attentional CNN Regression loss, Crystal loss, Center loss RGB S
CNN-LSP TIFS 2019 1D-CNN Trained with SVM RGB D
DT-Mask 2019 VGG16 Binary CE loss, Channel&Spatial discriminability RGB+OF D
VGG+LBP 2019 VGG16 Binary CE loss RGB S
CNN+OVLBP 2019 VGG16 Binary CE loss, NN classifier RGB S
HOG-Pert. 2019 Multi-scale CNN Binary CE loss RGB+HOG S
LBP-Pert. 2020 Multi-scale CNN Binary CE loss RGB+LBP S
TransRPPG SPL 2021 Vision Transformer Binary CE loss rPPG map D
Method Year Backbone Loss Input Static/Dynamic
CNN1 2014 8-layer CNN Trained with SVM RGB S
LSTM-CNN 2015 CNN+LSTM Binary CE loss RGB D
SpoofNet 2015 2-layer CNN Binary CE loss RGB S
HybridCNN 2017 VGG-Face Trained with SVM RGB S
CNN2 2017 VGG11 Binary CE loss RGB S
Ultra-Deep 2017 ResNet50+LSTM Binary CE loss RGB D
FASNet 2017 VGG16 Binary CE loss RGB S
CNN3 2018 Inception, ResNet Binary CE loss RGB S
MILHP 2018 ResNet+STN Multiple Instances CE loss RGB D
LSCNN 2018 9 PatchNets Binary CE loss RGB S
LiveNet 2018 VGG11 Binary CE loss RGB S
MS-FANS 2018 AlexNet+LSTM Binary CE loss RGB S
DeepColorFAS 2018 5-layer CNN Binary CE loss RGB, HSV, YCbCr S
Siamese 2019 AlexNet Contrastive loss RGB S
FSBuster 2019 ResNet50 Trained with SVM RGB S
FuseDNG 2019 7-layer CNN Binary CE loss, Reconstruction loss RGB S
STASN CVPR 2019 ResNet50+LSTM Binary CE loss RGB D
TSCNN TIFS 2019 ResNet18 Binary CE loss RGB, MSR S
FAS-UCM 2019 MobileNetV2, VGG19 Binary CE loss, Style loss RGB S
SLRNN 2019 ResNet50+LSTM Binary CE loss RGB D
GFA-CNN 2019 VGG16 Binary CE loss RGB S
3DSynthesis 2019 ResNet15 Binary CE loss RGB S
CompactNet NC 2020 VGG19 Points-to-Center triplet loss RGB S
SSR-FCN TIFS 2020 FCN with 6 layers Binary CE loss RGB S
FasTCo 2020 ResNet50 or MobileNetV2 Multi-class CE loss, Temporal Consistency loss, Class Consistency loss RGB D
DRL-FAS TIFS 2020 ResNet18+GRU Binary CE loss RGB S
SfSNet 2020 6-layer CNN Binary CE loss Albedo, Depth, Reflection S
LivenesSlight 2020 6-layer CNN Binary CE loss RGB S
MotionEnhancement 2020 VGGface+LSTM Binary CE loss RGB D
CFSA-FAS 2020 ResNet18 Binary CE loss RGB S
MC-FBC 2020 VGG16, ResNet50 Binary CE loss RGB S
SimpleNet 2020 Multi-stream 5-layer CNN Binary CE loss RGB, OF, RP D
PatchCNN 2020 SqueezeNet v1.1 Binary CE loss, Triplet loss RGB S
FreqSpatialTempNet 2020 ResNet18 Binary CE loss RGB, HSV, Spectral D
ViTranZFAS 2020 Vision Transformer Binary CE loss RGB S
CIFL TIFS 2021 ResNet18 Binary focal loss, camear type loss RGB S
Method Year Supervision Backbone Input Static/Dynamic
Depth&Patch IJCB 2017 Depth PatchNet, DepthNet YCbCr, HSV S
Auxiliary CVPR 2018 Depth, rPPG spectrum DepthNet RGB, HSV D
BASN ICCVW 2019 Depth, Reflection DepthNet, Enrichment RGB, HSV S
DTN CVPR 2019 BinaryMask Tree Network RGB, HSV S
PixBiS ICB 2019 BinaryMask DenseNet161 RGB S
A-PixBiS 2020 BinaryMask DenseNet161 RGB S
Auto-FAS ICASSP 2020 BinaryMask NAS RGB S
MRCNN 2020 BinaryMask Shallow CNN RGB S
FCN-LSA 2020 BinaryMask DepthNet RGB S
CDCN CVPR 2020 Depth DepthNet RGB S
FAS-SGTD CVPR 2020 Depth DepthNet, STPM RGB D
TS-FEN 2020 Depth ResNet34, FCN RGB, YCbCr, HSV S
SAPLC 2020 TernaryMap DepthNet RGB, HSV S
BCN ECCV 2020 BinaryMask, Depth, Reflection DepthNet RGB S
Disentangled ECCV 2020 Depth, TextureMap DepthNet RGB S
AENet ECCV 2020 Depth, Reflection ResNet18 RGB S
3DPC-Net 2020 3D Point Cloud ResNet18 RGB S
PS TBIOM 2020 BinaryMask or Depth ResNet50 or CDCN RGB S
NAS-FAS PAMI 2020 BinaryMask or Depth NAS RGB D
DAM 2021 Depth VGG16, TSM RGB D
Bi-FPNFAS 2021 Fourier spectra EfficientNetB0, FPN RGB S
DC-CDN IJCAI 2021 Depth CDCN RGB S
DCN IJCB 2021 Reflection DepthNet RGB S
LMFD-PAD 2021 BinaryMask Dual-ResNet50 RGB + frequency map S
MPFLN ICCVW 2021 Depth, BinaryMask CDCN, 3D-CDCN RGB S, D
Method Year Supervision Backbone Input Static/Dynamic
De-Spoof ECCV 2018 Depth, BinaryMask, FourierMap DSNet, DepthNet RGB, HSV S
Reconstruction 2019 RGB Input (live), ZeroMap (spoof) U-Net RGB S
LGSC 2020 ZeroMap (live) U-Net, ResNet18 RGB S
TAE ICASSP 2020 Binary CE loss, Reconstruction loss Info-VAE, DenseNet161 RGB S
STDN ECCV 2020 BinaryMask, RGB Input (live) U-Net, PatchGAN RGB S
GOGen CVPR 2020 RGB input DepthNet RGB+one-hot vector S
PhySTD 2021 Depth, RGB Input (live) U-Net, PatchGAN Frequency Trace S
MT-FAS PAMI 2021 ZeroMap (live), LearnableMap (Spoof) DepthNet RGB S
IF-OM 2021 RGB input, mixed input features MobileNetV2 + UNet RGB, mixed RGB, folded RGB S
Dual-Stage Disentanglement WACV 2021 ZeroMap (live), RGB Input for reconstruction U-Net, ResNet18 RGB S
Method Year Backbone Loss Static/Dynamic
OR-DA TIFS 2018 AlexNet Binary CE loss, MMD loss S
DTCNN 2019 AlexNet Binary CE loss, MMD loss S
Adversarial ICB 2019 ResNet18 Triplet loss, Adversarial loss S
ML-MMD ICMEW 2019 Multi-scale FCN CE loss, MMD loss S
OCA-FAS NC 2020 DepthNet Binary CE loss, Pixel-wise binary loss S
DR-UDA TIFS 2020 ResNet18 Center&Triplet loss, Adversarial loss, Disentangled loss S
DGP ICASSP 2020 DenseNet161 Feature divergence measure, BinaryMask loss S
Distillation J-STSP 2020 AlexNet Binary CE loss, MMD loss , Paired Similarity S
SCNN++PL+TC TIP 2021 ResNet18 CE Loss in labeled and unlabeled sets D
USDAN PR 2021 ResNet18 Adaptive binary CE loss, Entropy loss, Adversarial loss S
SASA 2021 ResNet18 CE Loss, Adversarial loss, Less-forgetting constraints, Contrastive semantic alignment S
Method Year Backbone Loss Static/Dynamic
MADDG CVPR 2019 DepthNet Binary CE & Depth loss, Multi-adversarial loss, Dual-force Triplet loss S
PAD-GAN CVPR 2020 ResNet18 Binary CE & Depth loss, Multi-adversarial loss, Dual-force Triplet loss S
DASN 2020 ResNet18 Binary CE & Spoof-irrelevant factor loss S
SSDG CVPR 2020 ResNet18 Binary CE loss, Single-Side adversarial loss, Asymmetric Triplet loss S
RF-Meta AAAI 2020 DepthNet Binary CE loss, Depth loss S
CCDD CVPRW 2020 ResNet50+LSTM Binary CE loss, Class-conditional loss D
SDA AAAI 2021 DepthNet Binary CE & Depth loss, Reconstruction loss, Orthogonality regularization S
D2AM AAAI 2021 DepthNet Binary CE loss, Depth loss, MMD loss S
DRDG IJCAI 2021 DepthNet Binary CE loss, Depth loss, Domain loss S
PDL-FAS 2021 DepthNet Binary CE loss, Depth loss S
AFNM+DCC ACMMM 2021 DepthNet Binary CE loss, Depth loss, Inter-Domain Compatible Loss, Inter-Class Separable Loss S
HFN+MP 2021 Two-stream ResNet50 Binary CE loss, MSE loss S
Method Year Backbone Loss Input
DTN CVPR 2019 Deep Tree Network Binary CE loss, Pixel-wise binary loss, Unsupervised Tree loss RGB, HSV
AIM-FAS AAAI 2020 DepthNet Depth loss, Contrastive Depth loss RGB
CM-PAD IJCB 2021 DepthNet, ResNet Binary CE loss, Depth loss, Gradient alignment RGB
Method Year Backbone Loss Input
AE+LBP 2018 AutoEncoder Reconstruction loss RGB
Anomaly 2019 ResNet50 Triplet focal loss, Metric-Softmax loss RGB
Anomaly2 2019 GoogLeNet or ResNet50 Mahalanobis distance RGB
Hypersphere 2020 ResNet18 Hypersphere loss RGB, HSV
Ensemble-Anomaly 2020 GoogLeNet or ResNet50 Gaussian Mixture Model (not end-to-end) RGB, patches
MCCNN 2020 LightCNN Binary CE loss, Contrastive loss Grayscale, IR, Depth, Thermal
End2End-Anomaly 2020 VGG-Face Binary CE loss, Pairwise confusion RGB
ClientAnomaly PR 2020 ResNet50 or GoogLeNet or VGG16 One-class SVM or Mahalanobis distance or Gaussian Mixture Model RGB

Method Year Backbone Loss Input Static/Dynamic
Thermal-FaceCNN 2019 AlexNet Regression loss Thermal infrared face image S
SLNet 2019 17-layer CNN Binary CE loss Stereo (left&right) face images S
Aurora-Guard 2019 U-Net Binary CE loss, Depth regression, Light Regression Casted face with dynamic changing light specified by random light CAPTCHA D
LFC 2019 AlexNet Binary CE loss Ray difference/microlens images from light field camera S
PAAS 2020 MobileNetV2 Contrastive loss, SVM Four-directional polarized face image S
Face-Revelio 2020 Siamese-AlexNet L1 distance Four flash lights displayed on four quarters of a screen D
SpecDiff 2020 ResNet4 Binary CE loss Concatenated face images w/ and w/o flash S
MC-PixBiS 2020 DenseNet161 Binary mask loss SWIR images differences S
Thermalization 2020 YOLO V3+GoogLeNet Binary CE loss Thermal infrared face image S
DP Bin-Cls-Net 2021 Shallow U-Net + Xception Transformation consistency, Relative disparity loss, Binary CE loss DP image pair S
Method Year Backbone Loss Input Fusion
FaceBagNet 2019 Multi-stream CNN Binary CE loss RGB, Depth, NIR face patches Feature-level
FeatherNets 2019 Ensemble-FeatherNet Binary CE loss Depth, NIR Decision-level
Attention 2019 ResNet18 Binary CE loss, Center loss RGB, Depth, NIR Feature-level
mmfCNN ACMMM 2019 ResNet34 Binary CE loss, Binary Center Loss RGB, NIR, Depth, HSV, YCbCr Feature-level
MM-FAS 2019 ResNet18/50 Binary CE loss RGB, NIR, Depth Feature-level
AEs+MLP 2019 Autoencoder, MLP Binary CE loss, Reconstruction loss Grayscale-Depth-Infrared composition Input-level
SD-Net 2019 ResNet18 Binary CE loss RGB, NIR, Depth Feature-level
Dual-modal 2019 MoblienetV3 Binary CE loss RGB, IR Feature-level
Parallel-CNN 2020 Attentional CNN Binary CE loss Depth, NIR Feature-level
Multi-Channel Detector 2020 RetinaNet (FPN+ResNet18) Landmark regression, Focal loss Grayscale-Depth-Infrared composition Input-level
PSMM-Net 2020 ResNet18 Binary CE loss for each stream RGB, Depth, NIR Feature-level
PipeNet 2020 SENet154 Binary CE loss RGB, Depth, NIR face patches Feature-level
MM-CDCN 2020 CDCN Pixel-wise binary loss, Contrastive depth loss RGB, Depth, NIR Feature&Decision-level
HGCNN 2020 Hypergraph-CNN, MLP Binary CE loss RGB, Depth Feature-level
MCT-GAN 2020 CycleGAN, ResNet50 GAN loss, Binary CE loss RGB, NIR Input-level
D-M-Net 2021 ResNeXt Binary CE loss Multi-preprocessed Depth, RGB-NIR composition Input&Feature-level
CMFL CVPR 2021 DenseNet161 Binary CE loss, Cross modal focal loss RGB, Depth Feature-level
MA-Net TIFS 2021 CycleGAN, ResNet18 Binary CE loss, GAN loss RGB, NIR Feature-level

Citation

If you find our work useful in your research, please consider citing:

@article{yu2021deep,
  title={Deep Learning for Face Anti-Spoofing: A Survey},
  author={Yu, Zitong and Qin, Yunxiao and Li, Xiaobai and Zhao, Chenxu and Lei, Zhen and Zhao, Guoying},
  journal={arXiv preprint arXiv:2106.14948},
  year={2021}
}