Point-based Multi-view Stereo Network

Introduction

PointMVSNet is a deep point-based deep framework for multi-view stereo (MVS). PointMVSNet directly processes the target scene as point clouds and predicts the depth in a coarse-to-fine manner. Our network leverages 3D geometry priors and 2D texture information jointly and effectively by fusing them into a feature-augmented point cloud, and processes the point cloud to estimate the 3D flow for each point. If you find this project useful for your research, please cite:

@InProceedings{ChenPMVSNet2019ICCV,
    author = {Chen, Rui and Han, Songfang and Xu, Jing and Su, Hao},
    title = {Point-based Multi-view Stereo Network},
    booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
    year = {2019}
}

How to use

Environment

The environment requirements are listed as follows:

Pytorch 1.0.1
CUDA 9.0
CUDNN 7.4.2
GCC5

Installation

Check out the source code

git clone https://github.com/callmeray/PointMVSNet && cd PointMVSNet
Install dependencies

bash install_dependencies.sh
Compile CUDA extensions

bash compile.sh

Training

Download the preprocessed DTU training data from MVSNet and unzip it to data/dtu.
Train the network

python pointmvsnet/train.py --cfg configs/dtu_wde3.yaml

You could change the batch size in the configuration file according to your own pc.

Testing

Download the rectified images from DTU benchmark and unzip it to data/dtu/Eval.
Test with your own model

python pointmvsnet/test.py --cfg configs/dtu_wde3.yaml
Test with the pretrained model

python pointmvsnet/test.py --cfg configs/dtu_wde3.yaml TEST.WEIGHT outputs/dtu_wde3/model_pretrained.pth

Depth Fusion

PointMVSNet generates per-view depth map. We need to apply depth fusion tools/depthfusion.py to get the complete point cloud. Please refer to MVSNet for more details.

eddienewton/PointMVSNet