Landmark Recurrent Network: An efficient and robust framwork for Deepfakes detection
The implementation for paper: Improving the Efficiency and Robustness for Deepfakes Detection through Precise Geometric Features (CVPR2021).
At current version, it is still the demo code of our proposed framework LRNet. It reproduces the whole procedure from a input video sample to its corresponding predicted label. We will update soon to optimize some components of it.
input
The input sample videos. Here we provide a real and a fake video.
landmarks
The extracted landmarks. Each video is corresponding to a .txt file.
model_weights
The two-stream RNN's weight.
visualize
The visualize results of landmrks extraction. The original landmarks detection (without calibration) is shown in "_origin.avi", and the calibrated one is shown in ".track.avi".
extract_landmarks.py
Perform the landmarks extraction and calibration.
landmark_utils.py
Some utils of landmarks extraction, Kalman filter, landmarks alignment and so on.
calib_utils.py
Some utils of calibration module, including the LK operation and so on.
classify.py
Perform the classification on extracted landmarks. Including reading and embedding the data, building the two-stream RNN and inference the samples.
shape_predictor_68_face_landmarks.dat
DLIB landmarks detector parameters. (Need to download MANUALLY.)
- argparse
- numpy
- python-opencv
- tqdm
- imutils
- dlib
- CPU version is sufficient.
- We do not include the necessary detector model (~100.7Mb). You may download it at http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2
- numpy
- tqdm
- TensorFlow >2.0
- CPU version is sufficient
To extract the landmarks from input videos, please ensure that all the samples are put in ./input
folder and run
python extract_landmarks.py
You can also specific your own input videos path by -i
or disable the visualization by -v
.
For detailed, please consult
python extract_landmarks.py -h
- Noted that the generated landmarks and visualized detection videos are put in the same folder as the code files, i.e. the "./", which cannot be specified.
- We have provided generated landmarks extraction results and visualization videos in
./landmarks
and./visualize
. Also you can delete these folders and regenerate them by yourself.
Before classification, make sure that the landmarks have been extracted and put in ./landmarks
folder.
Execute the following command:
python classify.py
and you can see the prediction results (predicted label and score of the corresponding sample. Noticed the score that "0=real" and "1=fake".
- Upload the demo (2021/3/29)
- Update the face/landmark detector.
- Update the calibration module.
- Update the pyramidal LK with numpy and openCV. (2021/4/14)
- Implement the pyramidal LK with openCV API. (2021/5/18, LATEST)
- Optimize the speed.
- Optimize the RNN part.
- 2021/5/18
- Updated the model weights. Now we provide weights trained on FF++ (./model_weights/ff)
and Deeperforensics-1.0 (./model_weights/deeper). We use
deeper
by default, because the given examples are from this dataset. You can change toff
inclassify.py
L107--L113. - Provide faster LK tracker. We now use openCV optical api to achieve LK track.
The Numpy version may cause some problems in
np.linalg.inv
, and we plan to fix it on June. - Update GPU support. We restrict Tensorflow not to use up the memory of GPU when it is supported. Although CPU is enough for inference. The GPU memory occupation may be < 1G.
- Updated the model weights. Now we provide weights trained on FF++ (./model_weights/ff)
and Deeperforensics-1.0 (./model_weights/deeper). We use
This is only a demo to show the whole detection framework, thus several components of our codes need optimization, and we are continuing to organize the code. The calibration module will be further designed as an API to calibrate the landmarks extracted by any other landmark detectors.