Paper | Supplementary Material
Official code for the IJCAI 2023 paper "VGOS: Voxel Grid Optimization for View Synthesis from Sparse Inputs".
git clone git@github.com:SJoJoK/VGOS.git
cd VGOS
pip install -r requirements.txt
Pytorch and torch_scatter installation is machine dependent, please install the correct version for your machine.
We conduct our experiments with torch 1.13.0+cu117 and torch-scatter 2.1.0+pt113cu117 on a single NVIDIA GeForce RTX 3090 GPU (24 GB).
(click to expand)
data
├── nerf_synthetic # Link: https://drive.google.com/drive/folders/128yBriW1IG_3NJ5Rp7APSTZsJqdJdfc1
│ └── [chair|drums|ficus|hotdog|lego|materials|mic|ship]
│ ├── [train|val|test]
│ │ └── r_*.png
│ └── transforms_[train|val|test].json
│
└── nerf_llff_data # Link: https://drive.google.com/drive/folders/14boI-o5hGO9srnWaaogTU5_ji7wkX2S7
└── [fern|flower|fortress|horns|leaves|orchids|room|trex]
├── poses_bounds.npy
└── [images_2|images_4]
-
Training
The
PSNR,SSIMandLPIPSof the testset will be evaluated after training.$ python run.py --config configs/llff_3v/room.py --render_test_get_metric
-
Render w/o re-training
$ python run.py --config configs/llff_3v/room.py --render_only --render_video
-
Re-conduct the experiments in the paper
$ zsh train_script/llff/llff_3v.sh $ zsh train_script/blender/blender_4v.sh
Feel free to modify the train scripts to suit your own needs, such as adding experiments under different training views or tuning parameters (please refer to the comments in configs/default.py for more information about the parameters).
Please note that the random selection of training views may result in fluctuations in performance. Therefore, the metrics presented for comparisons in our paper are the average scores of five experiments with different viewpoint samples. It is common to observe results that are slightly different, either higher or lower, than those presented in the paper.
The code is heavily based on DVGOv2 implementation, and the computation of depth smoothness loss is modified from RegNeRF. Thanks for sharing!
If you find this repo is helpful, please cite:
@InProceedings{sun2023vgos,
title={VGOS: Voxel Grid Optimization for View Synthesis from Sparse Inputs},
author={Jiakai Sun and Zhanjie Zhang and Jiafu Chen and Guangyuan Li and Boyan Ji and Lei Zhao and Wei Xing},
year={2023},
journal={arXiv preprint arXiv:2304.13386}
}