- Release training and testing code of DF_MVR;
- Release pretrained weight for pixel face dataset;
- Release pretrained weight for custom data;
RTX 3090 & RTX 2080 Ti
For 3090: We recommend CUDA 11.1 & Python v3.8.10 & Pytorch 1.9.0
conda create -n dfnet python=3.8.10
source activate dfnet
conda install -c conda-forge cudatoolkit=11.1 cudnn=8.1.0
conda install -c pytorch pytorch=1.9.0 torchvision
conda install tensorflow=2.5.0 numpy=1.20
pip install -r lib/requirement.txt
pytorch3d
involution
face-parsing
BFM Crop
Pytorch3d:
conda install -c fvcore -c iopath -c conda-forge fvcore iopath
conda install -c bottler nvidiacub
conda install pytorch3d=0.5.0 -c pytorch3d
Involution:
We simplified the code in the lib/involution directory, you don't need to download the source code. The following dependencies need to be installed:
pip install mmcv
pip install cupy-cuda111
We simplified the now_evaluation code in the 'lib/now_evaluation' directory. You needn't to download it.
Install mesh processing libraries from MPI-IS/mesh within the virtual environment.
cd lib/now_evaluation
pip install -r requirements.txt
git clone https://github.com/MPI-IS/mesh.git
cd mesh
conda install -c statiskit libboost-dev
BOOST_INCLUDE_DIRS=/path/to/boost/include make all
(Example: BOOST_INCLUDE_DIRS=/home/pointcloud/.conda/envs/dfnet/include/ make all)
Clone the flame-fitting repository and copy the required folders by the following comments
cd lib/now_evaluation
git clone https://github.com/Rubikplayer/flame-fitting.git
cp flame-fitting/smpl_webuser ./smpl_webuser -r
cp flame-fitting/sbody ./sbody -r
Clone Eigen and copy the it to the following folder
cd lib/now_evaluation
git clone https://gitlab.com/libeigen/eigen.git
cp eigen ./sbody/alignment/mesh_distance/eigen -r
Edit the file 'now_evaluation/sbody/alignment/mesh_distance/setup.py' to set EIGEN_DIR to the location of Eigen. Then compile the code by following command
cd lib/now_evaluation/sbody/alignment/mesh_distance
vim setup.py
(change c++11 to c++14, then save)
make
Please contact pixel-face for 'Pixel_Face' dataset
Download the face mask pretrain pth face_mask.pth into 'DF_MVR/pretrain/'
Download the BFM front face mat BFM_model_front.mat into 'lib/BFM/model_basis/BFM2009/'
python train.py
Download our newest DF_MVR pretrain pth 000000279.pth (RMSE 1.4040) into 'DF_MVR/pretrain/'
python test_pixel_face
python test_sample.py
If you find this work useful in your research, please cite:
@article{zhao20222d,
title={From 2D Images to 3D Model: Weakly Supervised Multi-View Face Reconstruction with Deep Fusion},
author={Zhao, Weiguang and Yang, Chaolong and Ye, Jianan and Yan, Yuyao and Yang, Xi and Huang, Kaizhu},
journal={arXiv preprint arXiv:2204.03842},
year={2022}
}