DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Cameras (ICCV 2021)

Yang Zheng*, Ruizhi Shao*, Yuxiang Zhang, Tao Yu, Zerong Zheng, Qionghai Dai, Yebin Liu.

This repository contains the official pytorch implementation of ”DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Cameras“.

News

[2021/9/26] We added more scans to MultiHuman dataset. You can use MultiHuman to train/fine-tune our model or your own models!
[2021/9/18] MultiHuman dataset for evaluation purpose is available!

Requirements

PyTorch
torchvision
trimesh
numpy
matplotlib
PIL
skimage
tqdm
cv2
json
taichi
taichi_three
taichi_glsl

Pretrained Model

We have provided pretrained model in One Drive

Training

To train DeepMultiCap model, please run the following code.

sh render_dataset.sh
sh trian.sh

Demo

Run the following code to inference on the MultiHuman dataset.

sh render_two.sh
sh demo.sh

Citation

@inproceedings{zheng2021deepmulticap,
title={DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Cameras},
author={Zheng, Yang and Shao, Ruizhi and Zhang, Yuxiang and Yu, Tao and Zheng, Zerong and Dai, Qionghai and Liu, Yebin},
booktitle={IEEE Conference on Computer Vision (ICCV 2021)},
year={2021},
}