Dual Variational Generation for Low Shot HFR

A PyTorch code of paper Dual Variational Generation for Low Shot Heterogeneous Face Recognition.

Our Heterogeneous Face Recognition Works

✨ DVG-Face: Dual Variational Generation for Heterogeneous Face Recognition

Chaoyou Fu, Xiang Wu, Yibo Hu, Huaibo Huang, and Ran He. IEEE TPAMI 2021

✨ Dual Variational Generation for Low Shot Heterogeneous Face Recognition

Chaoyou Fu, Xiang Wu, Yibo Hu, Huaibo Huang, and Ran He. NeurIPS 2019

✨ Towards Lightweight Pixel-Wise Hallucination for Heterogeneous Face Recognition

Chaoyou Fu, Xiaoqiang Zhou, Weizan He, and Ran He. IEEE TPAMI 2022

✨ Cross-Spectral Face Hallucination via Disentangling Independent Factors

Boyan Duan, Chaoyou Fu, Yi Li, Xingguang Song, and Ran He. CVPR 2020

News

The extension version of DVG is published in IEEE TPAMI 2021 (DVG-Face: Dual Variational Generation for Heterogeneous Face Recognition), and its code is released in https://github.com/BradyFU/DVG-Face. The newly released extension version has more powerful performances than this version.

Prerequisites

Python 2.7
Pytorch 0.4.1 & torchvision 0.2.1

Train the generator

Download LightCNN-29 model (Google Drive) pretrained on the MS-Celeb-1M dataset.
Train the generator:

sh run_train_generator.sh

Note that this is a simplified version of our original code:
1. The diversity loss and the adversarial loss in the paper are removed.
2. The distribution alignment loss is replaced by a Maximum Mean Discrepancy (MMD) loss.
The generated results during training will be saved in ./results.

Generate images from noise

Use the trained generator to sample 100,000 paired heterogeneous data:

Python val.py --pre_model './model/netG_model_epoch_50_iter_0.pth'

The generated fake NIR and VIS images will be saved in ./fake_images/nir_noise and ./fake_images/vis_noise, respectively.

Train the recognition model LightCNN-29

Use the real data and the generated fake data to train lightcnn:

sh run_train_lightcnn.sh

Performance

The performance on the 1-fold of CASIA NIR-VIS 2.0 dataset after running the above code:

Rank-1	VR@FAR=0.1%	VR@FAR=0.01%
99.9%	99.8%	98.9%

Citation

If you use our code for your research, please cite the following paper:

@article{fu2021dvg,
  title={DVG-face: Dual variational generation for heterogeneous face recognition},
  author={Fu, Chaoyou and Wu, Xiang and Hu, Yibo and Huang, Huaibo and He, Ran},
  journal={IEEE TPAMI},
  year={2021}
}

@inproceedings{fu2019dual,
  title={Dual Variational Generation for Low-Shot Heterogeneous Face Recognition},
  author={Fu, Chaoyou and Wu, Xiang and Hu, Yibo and Huang, Huaibo and He, Ran},
  booktitle={NeurIPS},
  year={2019}
}

@article{fu2022towards,
  title={Towards Lightweight Pixel-Wise Hallucination for Heterogeneous Face Recognition},
  author={Fu, Chaoyou and Zhou, Xiaoqiang and He, Weizan and He, Ran},
  journal={IEEE TPAMI},
  year={2022}
}

@inproceedings{duan2020cross,
  title={Cross-spectral face hallucination via disentangling independent factors},
  author={Duan, Boyan and Fu, Chaoyou and Li, Yi and Song, Xingguang and He, Ran},
  booktitle={CVPR},
  year={2020}
}