A Pytorch implementation of Instant-NSR, fast surface reconstructor as described in Human Performance Modeling and Rendering via Neural Animated Mesh.
Based on dense multi-view input, our approach enables efficient and high-quality reconstruction, compression, and rendering of human performances. It supports 4D photo-real content playback for various immersive experiences of human performances in virtual and augmented reality.This repo helps us to reconstruct 3D models from multi-view images in ~10 mins. Free to run our code!
First, you need to set training
pip install -r requirements.txt
# (optional) install the tcnn backbone
pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch
Tested on Ubuntu with torch 1.10 & CUDA 11.4 on RTX 3090.
We use the same data format as nerf and instant-ngp, and we provide a test dataset dance which is on google driver.
Please download and put it under {INPUTS}/dance
and then run our Instant-NSR code.
First time running will take some time to compile the CUDA extensions.
Train your own models, you can run following shell:
# Instant-NSR Training
CUDA_VISIBLE_DEVICES=${CUDA_DEVICE} python train_nerf.py "${INPUTS}/dance" --workspace "${WORKSAPCE}" --downscale 1 --network sdf
Then, you can extract surface from the trained network model by:
# Instant-NSR Mesh extraction
CUDA_VISIBLE_DEVICES=${CUDA_DEVICE} python train_nerf.py "${INPUTS}/dance" --workspace "${WORKSAPCE}" --downscale 1 --network sdf -mode mesh
Or, you can render target view with spefic camera view:
# Instant-NSR Rendering
CUDA_VISIBLE_DEVICES=${CUDA_DEVICE} python train_nerf.py "${INPUTS}/dance" --workspace "${WORKSAPCE}" --downscale 1 --network sdf -mode render
Here are some reconstruction results from our Instant-NSR code:
Our code is implemented on torch-ngp code base:
@misc{torch-ngp,
Author = {Jiaxiang Tang},
Year = {2022},
Note = {https://github.com/ashawkey/torch-ngp},
Title = {Torch-ngp: a PyTorch implementation of instant-ngp}
}