Color-NeuS: Reconstructing Neural Implicit Surfaces with Color

Licheng Zhong · Lixin Yang · Kailin Li · Haoyu Zhen · Mei Han . Cewu Lu

Project Page | arXiv | Data

demo_video.mp4

🚀 Dependencies

git clone https://github.com/Colmar-zlicheng/Color-NeuS.git
cd Color-NeuS
conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
pip install -r requirement.txt
pip install "git+https://github.com/facebookresearch/pytorch3d.git"

🚂 Train

Dataset

IHO Video
DTU (NeuS Preprocess Link | Raw Link)
BlendedMVS (NeuS Preprocess Link | Raw Link)
OmniObject3D

General command

set ${DATASET} as one in [iho, dtu, bmvs, omniobject3d]
set ${OBJECT_NAME} as the name of the object in the dataset

python train.py -g 0 --config configs/Color_NeuS_${DATASET}.yaml -obj ${OBJECT_NAME} --exp_id ${EXP_ID}

Command line arguments

-g, --gpu_id, visible GPUs for training, e.g. -g 0. Only supports single GPU.
--exp_id specify the name of experiment, e.g. --exp_id ${EXP_ID}. When --exp_id is provided, the code requires that no uncommitted change is remained in the git repo. Otherwise, it defaults to 'default' for training and 'eval_{cfg}_{OBJECT_NAME}' for evaluation. All results will be saved in exp/${EXP_ID}*{timestamp}.

For example

# IHO Video: ghost_bear
python train.py -g 0 --config configs/Color_NeuS_iho.yaml -obj ghost_bear --exp_id Color_NeuS_iho_ghost_bear
# DTU: dtu_scan83
python train.py -g 0 --config configs/Color_NeuS_dtu.yaml -obj 83 --exp_id Color_NeuS_dtu_83
# BlendedMVS: bmvs_bear
python train.py -g 0 --config configs/Color_NeuS_bmvs.yaml -obj bear --exp_id Color_NeuS_bmvs_bear
# OmniObject3D: doll_002
python train.py -g 0 --config configs/Color_NeuS_omniobject3d.yaml -obj doll_002 --exp_id Color_NeuS_omniobject3d_doll_002

Checkpoint

All the training checkpoints are saved at exp/${EXP_ID}_{timestamp}/checkpoints/

Other Method

We also provide our implementation of NeuS in this repo. To train NeuS, you can replace Color_NeuS_${DATASET}.yaml with NeuS_${DATASET}.yaml in the above command line, such as:

# IHO Video: ghost_bear
python train.py -g 0 --config configs/NeuS_iho.yaml -obj ghost_bear --exp_id NeuS_iho_ghost_bear

🧐 Inference

set corresponding ${DATASET} and ${OBJECT_NAME} as above
set ${PATH_TO_CHECKPOINT} as the path to the checkpoint (.pth.tar) to be loaded

python evaluation.py -g 0 --config configs/Color_NeuS_${DATASET}.yaml -obj ${OBJECT_NAME} -rr 512 --reload ${PATH_TO_CHECKPOINT}

-rr, --recon_res is the resolution of the reconstructed mesh. The default value is 512.

🔑 License

This code is available for non-commercial scientific research purposes as defined in the LICENSE file. By downloading and using the code you agree to the terms in the LICENSE.

🌏 Citation

@article{zhong2023colorneus,
    author    = {Zhong, Licheng and Yang, Lixin and Li, Kailin and Zhen, Haoyu and Han, Mei and Lu, Cewu},
    title     = {{Color-NeuS}: Reconstructing Neural Implicit Surfaces with Color},
    journal = {arXiv preprint arXiv:2308.06962},
    year      = {2023},
}

For more questions, please contact Licheng Zhong: zlicheng@sjtu.edu.cn

hirotong/Color-NeuS