/fake-face-detection

some collected paper and personal notes relevant to Fake Face Detetection

fake-face-detection

some collected paper and personal notes relevant to Fake Face Detetection

Challenge

Study

  1. [arXiv 2019] Deep Learning for Deepfakes Creation and Detection
  2. [ACM SIGSAC 2019] Poster: Towards Robust Open-World Detection of Deepfakes
  3. [arXiv 2020] DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection

I. Dataset

  1. FaceForensics++ Dataset

    • benchmark
    • paper: [ICCV 2019] FaceForensics++: Learning to Detect Manipulated Facial Images
      • 977 downloaded videos from youtube, 1000 original extracted sequences and its manipulated version
      • generated based on Deep-Fakes, Face2Face, FaceSwap and NeuralTextures

  2. [Google] DeepFakeDetection Dataset

    • homepage
      • over 363 original sequences from 28 paid actors in 16 different scenes
      • over 3000 manipulated videos using Deep-Fakes.

  3. DeepFake Forensics (Celeb-DF) Dataset

    • paper: [arXiv 2019] Celeb-DF: A New Dataset for DeepFake Forensics
      • real and DeepFake synthesized videos having similar visual quality on par with those circulated online
      • 408 original videos collected from YouTube with subjects of different ages, ethic groups and genders, and 795 DeepFake videos synthesized from these real videos.

  4. [Facebook] Deepfake Detection Challenge (DFDC) Dataset

  • paper : [arXiv 2019] The Deepfake Detection Challenge (DFDC) Preview Dataset

    • consisting of 5K videos featuring two facial modification algorithms.

    • a set of specific metrics to evaluate the performance have been defined and two existing models for detecting deepfakes have been tested to provide a reference perfor-mance baseline.

  1. TAMFA (Tampered face) Dataset

    • paper: [Expert Systems With Applications 2019] Face image manipulation detection based on a convolutional neural network
      • 8,950 facial images with unconstrained conditions such as pose, background cluttered, illumination change
      • 1,500 images labeled as “fake” and 7,450 images labeled as “normal”.

  2. SwapMe and FaceSwap Dataset

    • paper: [CVPRW 2017] Two-Stream Neural Networks for Tampered Face Detection
      • generated by using one iOS application called SwapMe and an open source face swap application called FaceSwap
      • contains 705 fake faces and 1,400 normal faces

  3. Deep Fakes Dataset

    • [to be released]
    • paper: [arXiv 2019] FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals
      • more ''in the wild" portrait videos
      • totaling up to 142 videos, 32 minutes, and 30 GBs

  4. Fake Faces in the Wild (FFW) Dataset

    • paper: [BIOSIG 2018] Fake Face Detection Methods: Can They Be Generalized?
      • more than 53,000 images (from 150 videos)

  5. Swapped Face Detection Dataset

  • [to be released]
  • paper: [arXiv 2019] Swapped Face Detection using Deep Learning and Subjective Assessment
    • A public dataset comprising 86 celebrities using 420,053 images.
    • This dataset is created using still images, different from other datasets created using video frames that may contain highly correlated images.

II. Current Work

(1) Special Artifact-Based

  1. [CVPRW 2019] Protecting World Leaders Against Deep Fakes
    • note;
    • capture the distinct facial expression and movements of a specific person use Action Unit (AU)
  2. [CVPRW 2019] Exposing DeepFake Videos By Detecting FaceWarping Artifacts
    • code; note;
    • improved version: DSP-FWA
    • current generated face have limited resolutions
  3. [WIFS 2018] In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking
    • code; note;
    • The lack of eye blinking indicates a synthesized video
  4. [ICASSP 2019] EXPOSING DEEP FAKES USING INCONSISTENT HEAD POSES
    • note; code;
    • the mismatch between the landmarks at center and outer contour of faked faces is revealed as inconsistent 3D head poses estimated from central and whole facial landmarks
  5. [arXiv 2019] FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals
    • note;
    • biological signals hidden in portrait videos can be used as an implicit descriptor of authenticity, because they are neither spatially nor temporally preserved in fake content.
  6. [WACVW 2019] Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations
    • code; note;
    • detect each manipulation method according to corresponding artifacts(eye color inconsistency, hard shadow in nose/contour, missing details in teeth, etc.)
  7. [ICCVW 2019] Deepfake Video Detection through Optical Flow Based CNN
    • we propose the adoption of optical flow fields to exploit possible inter-frame dissimilarities.
  8. [IMVOP 2018] Detection of Deepfake Video Manipulation
    • To contribute to a solution, photo response non uniformity (PRNU) analysis is tested for its effectiveness at detecting Deepfake video manipulation
  9. [arXiv 2019] Face X-ray for More General Face Forgery Detection
    • note
    • We observe that most existing face manipulation methods share a common step: blending the altered face into an existing background image.
    • The face X-ray of an input face image is a greyscale image that reveals whether the input image can be decomposed into the blending of two images from different sources.

(2) CNN-Based

  1. [ICCV 2019] FaceForensics++: Learning to Detect Manipulated Facial Images
  2. [ISITC 2018] Forensics Face Detection From GANs Using Convolutional Neural Network
    • note;
    • VGGFace + 2-way FN
  3. [ICASSP 2019] Capsule-forensics: Using Capsule Networks to Detect Forged Images and Videos
    • code; note;
    • image -> [face extracting & align] -> [VGG19] -> features -> [Capsule Network] -> fake/pristine
  4. [arXiv 2019] Swapped Face Detection using Deep Learning and Subjective Assessment
    • ResNet18 pretrained on ImageNet

(3) CNN+RNN

  1. [AVSS 2018] Deepfake Video Detection Using Recurrent Neural Networks
    • note;
    • CNN (InceptionV3) + LSTM
  2. [CVPR 2019] Recurrent Convolutional Strategies for Face Manipulation Detection in Videos
    • note;
    • CNN (DenseNet) + bidirectional RNN

(4) Two Stream

  1. [CVPRW 2017] Two-Stream Neural Networks for Tampered Face Detection
    • note;
    • Face Classification stream(GoogLeNet) + Patch Triplet stream(Steganalysis feature)
  2. [TIFS 2019] Attention-Based Two-Stream Convolutional Networks for Face Spoofing Detection
    • note;
    • RGB stream(contain texture details) + MSR stream(illumination invariant) & Attention-based fusion

(5) Auto-encoder

  1. [arXiv 2018] ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection
    • note;
    • input image -> [Encoder] -> Forensic Embedding -> [Decoder] -> reconstructed image
  2. [BTAS 2019] Multi-task Learning for Detecting and Segmenting Manipulated Facial Images and Videos
    • code; note;
    • multi-task learning of classification(real/fake) and segmentation(locating manipulated regions in images)
  3. [arXiv 2019] Towards Generalizable Forgery Detection with Locality-aware AutoEncoder
    • note
    • To bridge generalization gap, in this paper we propose Locality-aware AutoEn-coder (LAE), which combines fine-grained representation learning and enforcing locality in a unified frame-work.
    • A key characteristic of LAE is the augmented local interpretability, which could be regularized using extra pixel wise forgery masks, in order to learn intrinsic and meaningful forgery representations.

(6) Frequency Domain

  1. [arXiv 2019] Unmasking DeepFakes with simple Features
    • code; note
    • image -> [DFT] -> sinusoidal components of various frequencies -> [Azimuthal Average] -> 1D representation of FFT power spectrum -> [Classifier] -> Real/Fake

(7) General image manipulation

  1. [CVPR 2019] ManTraNet: Manipulation Tracing Network For Detection And Localization of Image Forgeries With Anomalous Features
    • code; note;
    • formulate the forgery localization problem as a local anomaly detection problem, design a Z-score feature to capture local anomaly, and propose a novel LSTM solution to assess local anomalies
  2. [arXiv 2019] Hybrid LSTM and Encoder-Decoder Architecture for Detection of Image Forgeries
    • note
    • This paper proposes a high-confidence manipulation localization architecture which utilizes resampling features, Long-Short Term Memory (LSTM) cells, and encoder-decoder network to segment out manipulated regions from non-manipulated ones

(8) Novel Network or Module

  1. [WIFS 2018] MesoNet: a Compact Facial Video Forgery Detection Network
    • code; note;
    • MesoNet exploits features at a meso-scopic level leveraging Inception Module and Dilated Convolution
  2. [Expert Systems With Applications 2019] Face image manipulation detection based on a convolutional neural network
    • note;
    • a customized convolutional neural network model for Manipulated Face (MANFA) & A hybrid framework (HF-MANFA) that uses Adaptive Boosting (AdaBoost) and eXtreme Gradient Boosting (XGBoost) to deal with the imbalanced dataset challenge
  3. [arXiv 2019] On the Detection of Digital Face Manipulation
    • note
    • proposed a novel attention-based layer to improve classification performance and produce an attention map indicating the manipulated facial regions.