/FACIAL

FACIAL: Synthesizing Dynamic Talking Face With Implicit Attribute Learning. ICCV, 2021.

Primary LanguagePythonGNU Affero General Public License v3.0AGPL-3.0

FACIAL: Synthesizing Dynamic Talking Face with Implicit Attribute Learning

PyTorch implementation for the paper:

FACIAL: Synthesizing Dynamic Talking Face with Implicit Attribute Learning

Chenxu Zhang, Yifan Zhao, Yifei Huang, Ming Zeng, Saifeng Ni, Madhukar Budagavi, Xiaohu Guo

ICCV 2021

[Paper] [Video] [Website]

Update: train a new person on Google Colab

Open In Colab

Run the test demo on Google Colab

Open In Colab

Requirements

  • Python environment
conda create -n audio_face
conda activate audio_face
  • ffmpeg
sudo apt-get install ffmpeg
  • python packages
pip install -r requirements.txt
  • you may add opencv by conda.
conda install opencv

Citation

@inproceedings{zhang2021facial,
  title={FACIAL: Synthesizing Dynamic Talking Face with Implicit Attribute Learning},
  author={Zhang, Chenxu and Zhao, Yifan and Huang, Yifei and Zeng, Ming and Ni, Saifeng and Budagavi, Madhukar and Guo, Xiaohu},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  pages={3867--3876},
  year={2021}
}

Acknowledgments

We use Deep3DFaceReconstruction for face reconstruction, DeepSpeech and VOCA for audio feature extraction, and 3dface for face rendering. Rendering-to-video module borrows heavily from everybody-dance-now.