/DeepPrivacy

DeepPrivacy: A Generative Adversarial Network for Face Anonymization

Primary LanguagePythonMIT LicenseMIT

DeepPrivacy

DeepPrivacy is a fully automatic anonymization technique for images.

This repository contains the source code for the paper "DeepPrivacy: A Generative Adversarial Network for Face Anonymization", published at ISVC 2019.

The DeepPrivacy GAN never sees any privacy sensitive information, ensuring a fully anonymized image. It utilizes bounding box annotation to identify the privacy-sensitive area, and sparse pose information to guide the network in difficult scenarios.

DeepPrivacy detects faces with state-of-the-art detection methods. Mask R-CNN is used to generate a sparse pose information of the face, and DSFD is used to detect faces in the image.

Citation

If you find this code useful, please cite the following:

@article{deep_privacy, 
         title={DeepPrivacy: A Generative Adversarial Network for Face Anonymization},
         url={https://arxiv.org/abs/1909.04538},
         journal={arXiv.org},
         author={Håkon Hukkelås and Rudolf Mester and Frank Lindseth},
         year={2019}
}

FDF Dataset

The FDF dataset will be released at github:hukkelas/FDF

Setting up your environment

Install the following:

  • Pytorch >= 1.0.0
  • Torchvision >= 0.3.0
  • NVIDIA Apex (Master branch)
  • Python >= 3.6

Then, install python packages:

pip install -r docker/requirements.txt

Docker

In our experiments, we use docker as the virtual environment.

Our docker image can be built by running:

cd docker/

docker build -t deep_privacy .

Then, training can be started with:

nvidia-docker run --rm  -it -v $PWD:/app  -e CUDA_VISIBLE_DEVICES=1 deep_privacy python -m deep_privacy.train models/default/config.yml

Config files

Hyperparameters and more can be set through config files, named config.yml.

From our paper, the following config files corresponds to our models

  • models/default/config.yml: Default 12M parameter model with pose (Max 256 channels in convolutions.)
  • models/no_pose/config.yml: Default 12M parameter model without pose
  • models/large/config.yml (BEST:): Default 46M parameter model with pose (Max 512 channels in convolutions). If you have the compute power, we recommend to use this model.
  • models/deep_discriminator/config.yml: Default deep discriminator model.

Pre-trained models

For each config file, you can download pre-trained models from the following URLS:

Automatic inference and anonymization of images

There are several scripts to perform inference

Every scripts require a path to a config.yml file. In these examples, we use the default model with 256 channels in the generator.

Download Face Detector: Before running inference, we expect that you have downloaded the DSFD face detection model, and place it to the path: deep_privacy/detection/dsfd/weights/WIDERFace_DSFD_RES152.pth. This can be downloaded from the official repository for DSFD [Google Drive Link].

Anonymizing a single image or folder

Run

python -m deep_privacy.inference.anonymize_folder model/default/config.yml --source_path testim.jpg --target_path testim_anonymized.jpg

Anonymizing Videos

Run

python -m deep_privacy.inference.anonymize_video model/default/config.yml --source_path path/to/video.mp4 --target_path path/to/video_anonymized.mp4

Note: DeepPrivacy is a frame-by-frame method, ensuring no temporal consistency in videos.

Anonymizing WIDER-Face Validation Datset

Run

python -m deep_privacy.inference.anonymize_wider models/default/config.yml --source_path path/to/Wider/face/dataset --target_path /path/to/output/folder

This expects the source path to include the following folders: WIDER_val and wider_face_split.

Calculate FID scores

  1. Generate real and fake images, where the last argument is the model config:
python -m deep_privacy.metrics.fid_official.calculate_fid models/default/config.yml
  1. Calculate FID with the official tensorflow code:
python deep_privacy/metrics/fid_official/calculate_fid_official.py models/default/fid_images/real models/default/fid_images/fake

Where the two last arguments are the paths to real and fake images.

NOTE: We use nvidias tensorflow docker container to run the FID code.: nvcr.io/nvidia/tensorflow:19.06-py3

Training your own model

Training your own model is easy. First, download our FDF dataset, and put it under data/fdf.

Then run:

python -m deep_privacy.train models/default/config.yml

License

All code is under MIT license, except the following:

Code under deep_privacy/detection: