krahets/awesome-mvs

A curated list of tutorials, papers, software related to multi-view stereo.

Awesome MVS

A curated list of tutorials, papers, software related to multi-view stereo.

Please visit awesome-computer-vision for a more generic computer vision list.

More multi-view stereo papers is available on Awesome-Learning-MVS, Awesome-PatchMatch-MVS, Awesome-MVS.

Feel free to create a Pull Request to add new papers!

Table of Contents

• Survey
• Paper
  • Multi-View Stereo
    • Depth-Map Representation
    • Volumetric Representation
    • Neural Implicit Representation
  • Mesh Texturing
  • Viewpoints and Trajectory Optimization
• Benchmark
• Open Source
• Commercial Software
• License

Tutorial

• Multi-View Stereo: A Tutorial. Y. Furukawa, C. Hernández. Foundations and Trends® in Computer Graphics and Vision, 2015.

• Multiple view geometry in computer vision. Hartley, Richard, and Andrew Zisserman. Cambridge university press, 2003.

Survey

• A comparison and evaluation of multi-view stereo reconstruction algorithms. Seitz, Steven M., et al. CVPR 2006.

• On benchmarking camera calibration and multi-view stereo for high resolution imagery. Strecha, Christoph, et al. CVPR 2008.

• State of the art in high density image matching. F. Remondino, M.G. Spera, E. Nocerino, F. Menna, F. Nex . The Photogrammetric Record 2014.

• Deep Learning for Multi-View Stereo via Plane Sweep: A Survey. Zhu, Qingtian, et al. arXiv 2021.

• Multi-view stereo in the Deep Learning Era: A comprehensive review. Wang, Xiang, et al. Displays 2021.

Paper

Multi-View Stereo

Depth-Map Representation

Geometry-Based Methods

• Multi-view stereo revisited. Goesele, Michael, Brian Curless, and Steven M. Seitz. CVPR 2006.
• Multi-view stereo for community photo collections. Goesele, Michael, et al. ICCV 2007.
• Using multiple hypotheses to improve depth-maps for multi-view stereo. Campbell, Neill DF, et al. ECCV 2008.
• Towards high-resolution large-scale multi-view stereo. Hiep, Vu Hoang, et al. CVPR 2009.
• Towards internet-scale multi-view stereo. Furukawa, Yasutaka, et al. CVPR 2010.
• (PMVS) Accurate, Dense, and Robust Multiview Stereopsis. Y. Furukawa, J. Ponce. CVPR 2007. PAMI 2010.
• Multi-view reconstruction preserving weakly-supported surfaces. Jancosek, Michal, and Tomás Pajdla. CVPR 2011.
• Efficient large-scale multi-view stereo for ultra high-resolution image sets. Tola, Engin, Christoph Strecha, and Pascal Fua. MVA 2012.
• (Similar to OpenMVS) Accurate multiple view 3d reconstruction using patch-based stereo for large-scale scenes. Shen, Shuhan. TIP 2013.
• MVE-A Multi-View Reconstruction Environment. Fuhrmann, Simon, Fabian Langguth, and Michael Goesele. GCH 2014.
• Patchmatch based joint view selection and depthmap estimation. Zheng, Enliang, et al. CVPR 2014.
• (Gipuma) Massively parallel multiview stereopsis by surface normal diffusion. Galliani, Silvano, Katrin Lasinger, and Konrad Schindler. ICCV 2015.
• (Colmap) Pixelwise View Selection for Unstructured Multi-View Stereo. J. L. Schönberger, E. Zheng, M. Pollefeys, J.-M. Frahm. ECCV 2016.
• TAPA-MVS: Textureless-aware patchmatch multi-view stereo. Romanoni, Andrea, and Matteo Matteucci. ICCV 2019.
• Pyramid multi‐view stereo with local consistency. Liao, Jie, et al. Computer Graphics Forum 2019.
• (ACMM) Multi-scale geometric consistency guided multi-view stereo. Qingshan Xu, and Wenbing Tao. CVPR 2019.
• (ACMP) Planar Prior Assisted PatchMatch Multi-View Stereo. Qingshan Xu, and Wenbing Tao. AAAI 2020.
• DP-MVS: Detail Preserving Multi-View Surface Reconstruction of Large-Scale Scenes. Liyang Zhou, et al. Remote Sens. 2021.

Surpervised Learning

• MVSNet: Depth Inference for Unstructured Multi-view Stereo, Y. Yao, Z. Luo, S. Li, T. Fang, L. Quan. ECCV 2018.

• DeepMVS: Learning Multi-View Stereopsis, Huang, P. and Matzen, K. and Kopf, J. and Ahuja, N. and Huang, J. CVPR 2018.

• MVDepthNet: Real-time multiview depth estimation neural network. Wang, Kaixuan, and Shaojie Shen. 3DV 2018.

• Recurrent MVSNet for high-resolution multi-view stereo depth inference. Yao, Yao, et al. CVPR 2019.

• DPSNet: End-to-end deep plane sweep stereo. Im, Sunghoon, et al. arXiv 2019.

• P-MVSNet: Learning patch-wise matching confidence aggregation for multi-view stereo. Luo, Keyang, et al. ICCV 2019.

• (PointMVSNet) Point-based Multi-view Stereo Network, Rui Chen, Songfang Han, Jing Xu, Hao Su. ICCV 2019.

• Pyramid multi-view stereo net with self-adaptive view aggregation. Yi, Hongwei, et al. ECCV 2020.

• (CasMVSNet) Cascade cost volume for high-resolution multi-view stereo and stereo matching. Gu, Xiaodong, et al. CVPR 2020.

• (CVP-MVSNet) Cost volume pyramid based depth inference for multi-view stereo. Yang, Jiayu, et al. CVPR 2020.

• Fast-MVSNet: Sparse-to-dense multi-view stereo with learned propagation and gauss-newton refinement. Yu, Zehao, and Shenghua Gao. CVPR 2020.

• (AttMVS) Attention-aware multi-view stereo. Luo, Keyang, et al. CVPR 2020.

• (Vis-MVSNet) Visibility-aware multi-view stereo network. Zhang, Jingyang, et al. BMVC 2020.

• Visibility-aware point-based multi-view stereo network. Chen, Rui, et al. PAMI 2020.

• PVSNet: Pixelwise visibility-aware multi-view stereo network. Xu, Qingshan, and Wenbing Tao. arXiv 2020.

• BP-MVSNet: Belief-propagation-layers for multi-view-stereo. Sormann, Christian, et al. 3DV 2020.

• DeepC-MVS: Deep confidence prediction for multi-view stereo reconstruction. Kuhn, Andreas, et al. 3DV 2020.

• Mesh-guided multi-view stereo with pyramid architecture. Wang, Yuesong, et al. CVPR 2020.

• PatchmatchNet: Learned multi-view patchmatch stereo. Wang, Fangjinhua, et al. CVPR 2021.

• AA-RMVSNet: Adaptive aggregation recurrent multi-view stereo network. Wei, Zizhuang, et al. ICCV 2021.

• PatchMatch-RL: Deep MVS with Pixelwise Depth, Normal, and Visibility. Lee, Jae Yong, et al. ICCV 2021.

• EPP-MVSNet: Epipolar-Assembling Based Depth Prediction for Multi-View Stereo. Ma, Xinjun, et al. ICCV 2021.

• Deep multi-view stereo gone wild. Darmon, François, et al. 3DV 2021.

• (GBiNet) Generalized Binary Search Network for Highly-Efficient Multi-View Stereo. Zhenxing Mi, Di Chang, Dan Xu. CVPR 2022.

• (UniMVSNet) Rethinking Depth Estimation for Multi-View Stereo: A Unified Representation. Rui Peng, et al. CVPR 2022.

• MVSTER: Epipolar Transformer for Efficient Multi-View Stereo. Xiaofen Wang, et al. ECCV 2022.

• TransMVSNet: Global Context-aware Multi-view Stereo Network with Transformers. Yikang Ding, et al. CVPR 2022.

Unsurpervised Learning

• Learning unsupervised multi-view stereopsis via robust photometric consistency. Khot, Tejas, et al. arXiv 2019.

• MVS2: Deep unsupervised multi-view stereo with multi-view symmetry. Dai, Yuchao, et al. 3DV 2019.

• Mˆ3VSNet: Unsupervised multi-metric multi-view stereo network. Huang, Baichuan, et al. ICIP 2021.

• Self-supervised multi-view stereo via effective co-segmentation and data-augmentation. Xu, Hongbin, et al. AAAI 2021.

• Self-supervised Learning of Depth Inference for Multi-view Stereo. Yang, Jiayu, Jose M. Alvarez, and Miaomiao Liu. CVPR 2021.

• Digging into Uncertainty in Self-supervised Multi-view Stereo. Xu, Hongbin, et al. ICCV 2021.

• RC-MVSNet: Unsupervised Multi-View Stereo with Neural Rendering. Di Chang, et al. arXiv 2022.

Volumetric Representation

Geometry-Based Methods

• (VRIP) A volumetric method for building complex models from range images. Curless, Brian, and Marc Levoy. PACMCGIT 1996.

• Reliable surface reconstruction from multiple range images. Hilton, Adrian, et al. ECCV 1996.

• Consensus surfaces for modeling 3D objects from multiple range images. Wheeler, Mark D., Yoichi Sato, and Katsushi Ikeuchi. ICCV 1998.

• A Theory of Shape by Space Carving. Kiriakos N. Kutulakos, S. Seitz. IJCV 2000.

• KinectFusion: Real-time dense surface mapping and tracking. Richard A. Newcombe, S. Izadi, Otmar Hilliges, D. Molyneaux, David Kim, A. Davison, P. Kohli, J. Shotton, Steve Hodges, A. Fitzgibbon. ISMAR 2011.

• (VoxelHashing) Real-time 3D reconstruction at scale using voxel hashing. M. Nießner, M. Zollhöfer, S. Izadi, M. Stamminger. ACM Trans. Graph 2013.

• ElasticFusion: Dense SLAM Without A Pose Graph. Thomas Whelan, et al. Robotics: Science and Systems 2015.

• BundleFusion: real-time globally consistent 3D reconstruction using on-the-fly surface re-integration. Angela Dai, M. Nießner, M. Zollhöfer, S. Izadi, C. Theobalt. TOGS 2016.

Learning-Based Methods

• Learning a multi-view stereo machine. Kar, Abhishek, Christian Häne, and Jitendra Malik. NeurIPS 2017.

• SurfaceNet: An end-to-end 3d neural network for multiview stereopsis. Ji, Mengqi, et al. ICCV 2017.

• RayNet: Learning Volumetric 3D Reconstruction with Ray Potentials, D. Paschalidou and A. O. Ulusoy and C. Schmitt and L. Gool and A. Geiger. CVPR 2018.

• Atlas: End-to-end 3d scene reconstruction from posed images. Murez, Zak, et al. ECCV 2020.

• RoutedFusion: Learning real-time depth map fusion. Weder, Silvan, et al. CVPR 2020.

• SurfaceNet+: An end-to-end 3D neural network for very sparse multi-view stereopsis. Ji, Mengqi, et al. PAMI 2020.

• NeuralRecon: Real-time coherent 3D reconstruction from monocular video. Sun, Jiaming, et al. CVPR 2021.

• NeuralFusion: Online depth fusion in latent space. Weder, Silvan, et al. CVPR 2021.

• PlanarRecon: Real-time 3D Plane Detection and Reconstruction from Posed Monocular Videos. Yuming Xie. CVPR 2022.

Neural Implicit Representation

• (DVR) Differentiable Volumetric Rendering: Learning implicit 3d representations without 3d supervision. Niemeyer, Michael, et al. CVPR 2020.

• (IDR) Multiview neural surface reconstruction by disentangling geometry and appearance. Yariv, Lior, et al. NeurIPS 2020.

• UNISURF: Unifying neural implicit surfaces and radiance fields for multi-view reconstruction. Oechsle, Michael, Songyou Peng, and Andreas Geiger. ICCV 2021.

• NeuS: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. Wang, Peng, et al. NeurIPS 2021.

• (VolSDF) Volume rendering of neural implicit surfaces. Yariv, Lior, et al. NeurIPS 2021.

• NerfingMVS: Guided optimization of neural radiance fields for indoor multi-view stereo. Wei, Yi, et al. ICCV 2021.

• (ManhattanSDF) Neural 3D Scene Reconstruction with the Manhattan-world Assumption. Haoyu Guo, et al. CVPR 2022.

• (NeuralRecon-W) Neural 3D Reconstruction in the Wild. Jiaming Sun, et al. SIGGRAPH 2022.

• Neural RGB-D Surface Reconstruction. Dejan Azinović, et al. CVPR 2022.

• SurRF: Unsupervised Multi-view Stereopsis by Learning Surface Radiance Field Jinzhi Zhang, et al. TPAMI 2022.

• MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction. Zehao Yu, et al. arXiv 2022.

Mesh Texturing

• Seamless image-based texture atlases using multi-band blending. C. Allène, J-P. Pons and R. Keriven. ICPR 2008.

• (mvs-texturing) Let There Be Color! - Large-Scale Texturing of 3D Reconstructions. M. Waechter, N. Moehrle, M. Goesele. ECCV 2014.

• Texture Mapping for 3D Reconstruction with RGB-D Sensor. Yanping Fu, Qingan Yan, Long Yang, Jie Liao, Chunxia Xiao. CVPR 2018.

Viewpoints and Trajectory Optimization

• Next Best View Planning for Active Model Improvement. Dunn, Enrique, and Jan-Michael Frahm. BMVC 2009.

• Receding Horizon "Next-Best-View" Planner for 3D Exploration. Andreas Bircher, et al. ICRA 2016.

• Submodular Trajectory Optimization for Aerial 3D Scanning. M. Roberts, A. Truong, D. Dey, S. Sinha, A. Kapoor, N. Joshi, P. Hanrahan. 2017.

• Aerial path planning for urban scene reconstruction: A continuous optimization method and benchmark. Smith, Neil, et al. 2018.

• Plan3D: Viewpoint and trajectory optimization for aerial multi-view stereo reconstruction. Hepp, Benjamin, Matthias Nießner, and Otmar Hilliges. ACM TOG 2018.

• Learn-to-Score: Efficient 3D Scene Exploration by Predicting View Utility. Benjamin Hepp, et al. ECCV 2018.

• Automatic and semantically-aware 3D UAV flight planning for image-based 3D reconstruction. Koch, Tobias, Marco Körner, and Friedrich Fraundorfer. Remote Sensing 2019.

• Next-Best View Policy for 3D Reconstruction. Daryl Peralta, et al. ECCV Workshop 2020.

• Offsite aerial path planning for efficient urban scene reconstruction. Zhou, Xiaohui, et al. ACM TOG 2020.

Benchmark

• DTU. Large scale multi-view stereopsis evaluation. Jensen, Rasmus, et al. CVPR 2014. Large-scale data for multiple-view stereopsis. Aanæs, Henrik, et al. ICCV2016.

• Tanks and Temples: Benchmarking large-scale scene reconstruction. Knapitsch, Arno, et al. ACM TOG 2017.

• ETH3D. A multi-view stereo benchmark with high-resolution images and multi-camera videos. Schops, Thomas, et al. CVPR 2017.

• ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes. Angela Dai, Angel X. Chang, M. Savva, Maciej Halber, T. Funkhouser, M. Nießner. CVPR 2017.

• BlendedMVS: A large-scale dataset for generalized multi-view stereo networks. Yao, Yao, et al. CVPR 2020.

• GigaMVS: A Benchmark for Ultra-large-scale Gigapixel-level 3D Reconstruction. Zhang, Jianing, et al. PAMI 2021.

• Multi-sensor large-scale dataset for multi-view 3D reconstruction. Oleg Voynov, et al. arXiv 2022.

• Capturing, Reconstructing, and Simulating: the UrbanScene3D Dataset. Liqiang Lin, et al. ECCV 2022.

Open Source

Project	Language	License
CMVS-PMVS	C++, CUDA	GPL
Colmap	C++, CUDA	BSD 3-Clause
Gipuma + Fusibile	C++, CUDA	GPL-3.0
MeshRoom(AliceVision)	C++, Python	MPL2
MVE	C++	BSD 3-Clause
OpenMVS	C++, CUDA(Optional)	AGPL3
MVS-Texturing	C++	BSD 3-Clause

Commercial Software

Software	Company
ContextCapture	Bentley Systems
DJI Terra	DJI
MetaShape	Agisoft
Pix4Dmapper	Pix4D
RealityCapture	Epic Games

License

MIT