DeepFake_Detection

What is DeepFake?

DeepFakes are images or videos which have been altered to feature the face of someone else, like an advanced form of Face Swapping, using an AI DeepFake Converter.
Many Deep Fakes are done by superimposing or combining existing images into source images and videos using Generative Adversarial Networks (GAN) and these networks are developing better every day

Demo of the Project

Impact of DeepFake Videos

DeepFakes can be used to create fake news, celebrity unusual videos, politician content videos, and financial fraud.
False Rumours can be spread using DeepFake videos which causes unrest and mental anxiety among people.
Many fields in Film Industry, content providers, and social media platforms are fighting against DeepFake.

Project Objectives:

Identification of deepfakes is necessary to prevent the use of malicious AI. We intend to,

Build a model that processes the given video and classifies it as REAL or FAKE.
Dploy a feature in the social media apps that can detect and give a warning to the content provider who is willing to do viral by uploading deepFaked images or videos.

Goal:

To Crate a deep learning model that is capable of recognizing deepfake images. A thorough analysis of deepfake video frames to identify slight imperfections in the face head and the model will learn what features differentiate a real image from a deepfake.

Project Pipeline

Steps	Dscription
Step1	Loading the datasets
Step2	Extracting videos from the dataset
Step3	Extract all frames in the video for both real and fake
Step4	Recognize the face subframe
Step5	Locating the facial landmarks
Step6	Frame-by-frame analysis to address any changes in the face landmarks
Step7	To Classify the video either as REAL or Fake.

General WorkFlow:

Pre-processing:

Prediction WorkFlow:

Models Usage:

Models with CNN Architecture

Implemented the following models with CNN architecture MesoNet

This model is pre-trained to detect deepfake images, but it is bad at detecting Fake video frames ResNet50v
This model is trained using dee fake images cropped from the videos with preset weights of imagenet dataset EfficientNetB0
This model is also trained using deepfake images cropped from the videos with preset weights of imagenet dataset

Models with CNN + Seqential Architecture

InceptionV3(CNN Model) + GRU(sequential)

This model works well because of both CNN and Sequential architecture.
Test Accuracy is approx. 82%
For Each Frame in the Video, it will generate the feature Vectors
HyperParameters used:
Optimizer: Adam ( Adam Works fine as it changes the Learning Rate over time )
Metric as Accuracy
loss as sparse_categorical_crossentropy (loss function when there are two or more label classes )
Among all the Optimizers Adam is Working Well.
The accuracy of the model increases as the epochs are increasing.

Limitations This model doesn’t work well when there are multiple faces in the Video, as it needs to detect the multiple faces in each Frame.

EfficientNetB2(CNN Model) + GRU(sequential)

This model works well because of both CNN and Sequential architecture
Test Accuracy is approx. 85%
For Each Frame in the Video, it will generate the feature Vectors
HyperParameters used:
Optimizer: Adam ( Adam Works fine as it changes the Learning Rate over time )
Metric as Accuracy
loss as sparse_categorical_crossentropy (loss function when there are two or more label classes )
Among all the Optimizers Adam is Working Fine.
The accuracy of the model increases as the epochs are increasing. Limitations
This model doesn’t work well when there is dark background in the video frames. As it is difficult to detect the faces in the Video Frame.

Running Code

Combination of CNN and RNN model is used to detect Fake Videos. We achieved a test accuracy ~85% on sample DFDC dataset
To run this code first run this command.

  pip install -r requirements.txt

Run main.py file in deploy folder

  python main.py

Make sure the required packages are installed, and it is preferred to run on GPU. The results are given in about a minute for a 10 second 30fps video.

Languages and Tools:

Conclusion:

In this project, we have implemented a method for the detection of Deep-Fake videos using the combination of CNN and RNN architecture. We have kept our focus on Face-Swapped Deep-Fake videos.
We primarily experimented only with various pre-trained CNN models like EfficientNet, and ResNet by finding the probability of each video frame being fake and predicting the output based on an aggregate of these probabilities. But the results weren’t satisfactory, so we went forward by combining CNN and RNN models.
For the CNN + RNN model, the features of face-cropped video frames are extracted using pre�trained CNN models and it is passed onto the RNN model which classifies the video as REAL or FAKE. We Experimented with EfficientNet and inception net for the feature extraction part and GRU is used to make the classification. We have obtained a maximum Test Accuracy of ~85% using this approach. Our model has high precision for FAKE videos which is obtained by giving more FAKE videos during the training of the Model.

manoj24vvr/DeepFake_Detection