Official implementation of Fast Generalizable Gaussian Splatting Reconstruction from Multi-View Stereo
Tianqi Liu1, Guangcong Wang2,3, Shoukang Hu2, Liao Shen1, Xinyi Ye1, Yuhang Zang4, Zhiguo Cao1, Wei Li2, Ziwei Liu2
1Huazhong University of Science and Technology 2Nanyang Technological University
3Great Bay University 4Shanghai AI Laboratory
Project page | Paper | Youtube
Coming soon!
TL;DR: MVSGaussian is a Gaussian-based method designed for efficient reconstruction of unseen scenes from sparse views in a single forward pass. It offers high-quality initialization for fast training and real-time rendering.
We present MVSGaussian, a new generalizable 3D Gaussian representation approach derived from Multi-View Stereo (MVS) that can efficiently reconstruct unseen scenes. Specifically, 1) we leverage MVS to encode geometry-aware Gaussian representations and decode them into Gaussian parameters. 2) To further enhance performance, we propose a hybrid Gaussian rendering that integrates an efficient volume rendering design for novel view synthesis. 3) To support fast fine-tuning for specific scenes, we introduce a multi-view geometric consistent aggregation strategy to effectively aggregate the point clouds generated by the generalizable model, serving as the initialization for per-scene optimization. Compared with previous generalizable NeRF-based methods, which typically require minutes of fine-tuning and seconds of rendering per image, MVSGaussian achieves real-time rendering with better synthesis quality for each scene. Compared with the vanilla 3D-GS, MVSGaussian achieves better view synthesis with less training computational cost. Extensive experiments on DTU, Real Forward-facing, NeRF Synthetic, and Tanks and Temples datasets validate that MVSGaussian attains state-of-the-art performance with convincing generalizability, real-time rendering speed, and fast per-scene optimization.
If you find our work useful for your research, please cite our paper.
@article{liu2024mvsgaussian,
title={Fast Generalizable Gaussian Splatting Reconstruction from Multi-View Stereo},
author={Liu, Tianqi and Wang, Guangcong and Hu, Shoukang and Shen, Liao and Ye, Xinyi and Zang, Yuhang and Cao, Zhiguo and Li, Wei and Liu, Ziwei},
journal={arXiv preprint arXiv:2405.12218},
year={2024}
}