KyrieLiu52/awesome-mvs-krahets

A curated list of tutorials, papers, software related to multi-view stereo.

Awesome MVS

A curated list of tutorials, papers, software related to multi-view stereo.

Please visit awesome-computer-vision for a more generic computer vision list.

Contents

• Tutorial
• Survey
• Paper
  • Multi-View Stereo
  • Mesh Texturing
  • Viewpoints and Trajectory Optimization
• Benchmark
• Open Source
• Commercial Software
• License
• Contribution

Tutorial

• Multi-View Stereo: A Tutorial. Y. Furukawa, C. Hernández. Foundations and Trends® in Computer Graphics and Vision, 2015.

• Multiple view geometry in computer vision. Hartley, Richard, and Andrew Zisserman. Cambridge university press, 2003.

Survey

• A comparison and evaluation of multi-view stereo reconstruction algorithms. Seitz, Steven M., et al. CVPR 2006.

• On benchmarking camera calibration and multi-view stereo for high resolution imagery. Strecha, Christoph, et al. CVPR 2008.

• State of the art in high density image matching. F. Remondino, M.G. Spera, E. Nocerino, F. Menna, F. Nex . The Photogrammetric Record 2014.

• Deep Learning for Multi-View Stereo via Plane Sweep: A Survey. Zhu, Qingtian, et al. arXiv 2021.

• Multi-view stereo in the Deep Learning Era: A comprehensive review. Wang, Xiang, et al. Displays 2021.

Paper

Multi-View Stereo

Depth Map & Point Cloud Methods

Stereo Matching

• Stereo Processing by Semiglobal Matching and Mutual Information. Hirschmuller, Heiko. PAMI 2007.

• Patchmatch stereo-stereo matching with slanted support windows. Bleyer, M., Rhemann, C., & Rother, C. BMVC 2011.

Geometry-Based

• Multi-view stereo revisited. Goesele, Michael, Brian Curless, and Steven M. Seitz. CVPR 2006.

• Multi-view stereo for community photo collections. Goesele, Michael, et al. ICCV 2007.

• Using multiple hypotheses to improve depth-maps for multi-view stereo. Campbell, Neill DF, et al. ECCV 2008.

• Towards high-resolution large-scale multi-view stereo. Hiep, Vu Hoang, et al. CVPR 2009.

• Towards internet-scale multi-view stereo. Furukawa, Yasutaka, et al. CVPR 2010.

• Accurate, Dense, and Robust Multiview Stereopsis. Y. Furukawa, J. Ponce. CVPR 2007. PAMI 2010.

• Multi-view reconstruction preserving weakly-supported surfaces. Jancosek, Michal, and Tomás Pajdla. CVPR 2011.

• DAISY: An efficient dense descriptor applied to wide-baseline stereo. Tola, Engin, Vincent Lepetit, and Pascal Fua. PAMI 2009.

• Efficient large-scale multi-view stereo for ultra high-resolution image sets. Tola, Engin, Christoph Strecha, and Pascal Fua. MVA 2012.

• Accurate multiple view 3d reconstruction using patch-based stereo for large-scale scenes. Shen, Shuhan. TIP 2013.

• MVE-A Multi-View Reconstruction Environment. Fuhrmann, Simon, Fabian Langguth, and Michael Goesele. GCH 2014.

• Patchmatch based joint view selection and depthmap estimation. Zheng, Enliang, et al. CVPR 2014.

• Massively parallel multiview stereopsis by surface normal diffusion. Galliani, Silvano, Katrin Lasinger, and Konrad Schindler. ICCV 2015.

• Pixelwise View Selection for Unstructured Multi-View Stereo. J. L. Schönberger, E. Zheng, M. Pollefeys, J.-M. Frahm. ECCV 2016.

• Multi-scale geometric consistency guided multi-view stereo. Xu, Qingshan, and Wenbing Tao. CVPR 2019.

• TAPA-MVS: Textureless-aware patchmatch multi-view stereo. Romanoni, Andrea, and Matteo Matteucci. ICCV 2019.

• Pyramid multi‐view stereo with local consistency. Liao, Jie, et al. Computer Graphics Forum 2019.

Surpervised Learning

• MVSNet: Depth Inference for Unstructured Multi-view Stereo, Y. Yao, Z. Luo, S. Li, T. Fang, L. Quan. ECCV 2018.

• DeepMVS: Learning Multi-View Stereopsis, Huang, P. and Matzen, K. and Kopf, J. and Ahuja, N. and Huang, J. CVPR 2018.

• MVDepthNet: Real-time multiview depth estimation neural network. Wang, Kaixuan, and Shaojie Shen. 3DV 2018.

• Recurrent MVSNet for high-resolution multi-view stereo depth inference. Yao, Yao, et al. CVPR 2019.

• DPSNet: End-to-end deep plane sweep stereo. Im, Sunghoon, et al. arXiv 2019.

• P-MVSNet: Learning patch-wise matching confidence aggregation for multi-view stereo. Luo, Keyang, et al. ICCV 2019.

• Point-based Multi-view Stereo Network, Rui Chen, Songfang Han, Jing Xu, Hao Su. ICCV 2019.

• Pyramid multi-view stereo net with self-adaptive view aggregation. Yi, Hongwei, et al. ECCV 2020.

• Cascade cost volume for high-resolution multi-view stereo and stereo matching. Gu, Xiaodong, et al. CVPR 2020.

• Cost volume pyramid based depth inference for multi-view stereo. Yang, Jiayu, et al. CVPR 2020.

• Fast-MVSNet: Sparse-to-dense multi-view stereo with learned propagation and gauss-newton refinement. Yu, Zehao, and Shenghua Gao. CVPR 2020.

• Attention-aware multi-view stereo. Luo, Keyang, et al. CVPR 2020.

• Visibility-aware multi-view stereo network. Zhang, Jingyang, et al. arXiv 2020.

• Visibility-aware point-based multi-view stereo network. Chen, Rui, et al. PAMI 2020.

• PVSNet: Pixelwise visibility-aware multi-view stereo network. Xu, Qingshan, and Wenbing Tao. arXiv 2020.

• BP-MVSNet: Belief-propagation-layers for multi-view-stereo. Sormann, Christian, et al. 3DV 2020.

• DeepC-MVS: Deep confidence prediction for multi-view stereo reconstruction. Kuhn, Andreas, et al. 3DV 2020.

• Mesh-guided multi-view stereo with pyramid architecture. Wang, Yuesong, et al. CVPR 2020.

• PatchmatchNet: Learned multi-view patchmatch stereo. Wang, Fangjinhua, et al. CVPR 2021.

• AA-RMVSNet: Adaptive aggregation recurrent multi-view stereo network. Wei, Zizhuang, et al. ICCV 2021.

• PatchMatch-RL: Deep MVS with Pixelwise Depth, Normal, and Visibility. Lee, Jae Yong, et al. ICCV 2021.

• EPP-MVSNet: Epipolar-Assembling Based Depth Prediction for Multi-View Stereo. Ma, Xinjun, et al. ICCV 2021.

• Deep multi-view stereo gone wild. Darmon, François, et al. 3DV 2021.

• Generalized Binary Search Network for Highly-Efficient Multi-View Stereo. Zhenxing Mi, Di Chang, Dan Xu. CVPR 2022.

• Rethinking Depth Estimation for Multi-View Stereo: A Unified Representation. Rui Peng, et al. CVPR 2022.

Unsurpervised Learning

• Learning unsupervised multi-view stereopsis via robust photometric consistency. Khot, Tejas, et al. arXiv 2019.

• MVS2: Deep unsupervised multi-view stereo with multi-view symmetry. Dai, Yuchao, et al. 3DV 2019.

• Mˆ3VSNet: Unsupervised multi-metric multi-view stereo network. Huang, Baichuan, et al. ICIP 2021.

• Self-supervised multi-view stereo via effective co-segmentation and data-augmentation. Xu, Hongbin, et al. AAAI 2021.

• Self-supervised Learning of Depth Inference for Multi-view Stereo. Yang, Jiayu, Jose M. Alvarez, and Miaomiao Liu. CVPR 2021.

• Digging into Uncertainty in Self-supervised Multi-view Stereo. Xu, Hongbin, et al. ICCV 2021.

• NerfingMVS: Guided optimization of neural radiance fields for indoor multi-view stereo. Wei, Yi, et al. ICCV 2021.

Volumetric & Implicit Methods

Geometry-Based

• A volumetric method for building complex models from range images. Curless, Brian, and Marc Levoy. PACMCGIT 1996.

• Reliable surface reconstruction from multiple range images. Hilton, Adrian, et al. ECCV 1996.

• Consensus surfaces for modeling 3D objects from multiple range images. Wheeler, Mark D., Yoichi Sato, and Katsushi Ikeuchi. ICCV 1998.

• A Theory of Shape by Space Carving. Kiriakos N. Kutulakos, S. Seitz. IJCV 2000.

• KinectFusion: Real-time dense surface mapping and tracking. Richard A. Newcombe, S. Izadi, Otmar Hilliges, D. Molyneaux, David Kim, A. Davison, P. Kohli, J. Shotton, Steve Hodges, A. Fitzgibbon. ISMAR 2011.

• Real-time 3D reconstruction at scale using voxel hashing. M. Nießner, M. Zollhöfer, S. Izadi, M. Stamminger. ACM Trans. Graph 2013.

• BundleFusion: real-time globally consistent 3D reconstruction using on-the-fly surface re-integration. Angela Dai, M. Nießner, M. Zollhöfer, S. Izadi, C. Theobalt. TOGS 2016.

Learning-Based

• Learning a multi-view stereo machine. Kar, Abhishek, Christian Häne, and Jitendra Malik. NeuralIPS 2017.

• SurfaceNet: An end-to-end 3d neural network for multiview stereopsis. Ji, Mengqi, et al. ICCV 2017.

• RayNet: Learning Volumetric 3D Reconstruction with Ray Potentials, D. Paschalidou and A. O. Ulusoy and C. Schmitt and L. Gool and A. Geiger. CVPR 2018.

• Atlas: End-to-end 3d scene reconstruction from posed images. Murez, Zak, et al. ECCV 2020.

• RoutedFusion: Learning real-time depth map fusion. Weder, Silvan, et al. CVPR 2020.

• SurfaceNet+: An end-to-end 3D neural network for very sparse multi-view stereopsis. Ji, Mengqi, et al. PAMI 2020.

• Differentiable Volumetric Rendering: Learning implicit 3d representations without 3d supervision. Niemeyer, Michael, et al. CVPR 2020.

• Multiview neural surface reconstruction by disentangling geometry and appearance. Yariv, Lior, et al. NeuralIPS 2020.

• NeuralRecon: Real-time coherent 3D reconstruction from monocular video. Sun, Jiaming, et al. CVPR 2021.

• NeuralFusion: Online depth fusion in latent space. Weder, Silvan, et al. CVPR 2021.

• UNISURF: Unifying neural implicit surfaces and radiance fields for multi-view reconstruction. Oechsle, Michael, Songyou Peng, and Andreas Geiger. ICCV 2021.

• NeuS: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. Wang, Peng, et al. arXiv 2021.

• Volume rendering of neural implicit surfaces. Yariv, Lior, et al. NeuralIPS 2021.

Mesh Texturing

• Seamless image-based texture atlases using multi-band blending. C. Allène, J-P. Pons and R. Keriven. ICPR 2008.

• Let There Be Color! - Large-Scale Texturing of 3D Reconstructions. M. Waechter, N. Moehrle, M. Goesele. ECCV 2014.

• Texture Mapping for 3D Reconstruction with RGB-D Sensor. Yanping Fu, Qingan Yan, Long Yang, Jie Liao, Chunxia Xiao. CVPR 2018.

Viewpoints and Trajectory Optimization

• Next Best View Planning for Active Model Improvement. Dunn, Enrique, and Jan-Michael Frahm. BMVC 2009.

• Submodular Trajectory Optimization for Aerial 3D Scanning. M. Roberts, A. Truong, D. Dey, S. Sinha, A. Kapoor, N. Joshi, P. Hanrahan. 2017.

• Aerial path planning for urban scene reconstruction: A continuous optimization method and benchmark. Smith, Neil, et al. 2018.

• Plan3D: Viewpoint and trajectory optimization for aerial multi-view stereo reconstruction. Hepp, Benjamin, Matthias Nießner, and Otmar Hilliges. ACM TOG 2018.

• Automatic and semantically-aware 3D UAV flight planning for image-based 3D reconstruction. Koch, Tobias, Marco Körner, and Friedrich Fraundorfer. Remote Sensing 2019.

• Offsite aerial path planning for efficient urban scene reconstruction. Zhou, Xiaohui, et al. ACM TOG 2020.

Benchmark

• DTU. Large scale multi-view stereopsis evaluation. Jensen, Rasmus, et al. CVPR 2014. Large-scale data for multiple-view stereopsis. Aanæs, Henrik, et al. ICCV2016.

• Tanks and Temples: Benchmarking large-scale scene reconstruction. Knapitsch, Arno, et al. ACM TOG 2017.

• ETH3D. A multi-view stereo benchmark with high-resolution images and multi-camera videos. Schops, Thomas, et al. CVPR 2017.

• ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes. Angela Dai, Angel X. Chang, M. Savva, Maciej Halber, T. Funkhouser, M. Nießner. CVPR 2017.

• BlendedMVS: A large-scale dataset for generalized multi-view stereo networks. Yao, Yao, et al. CVPR 2020.

• GigaMVS: A Benchmark for Ultra-large-scale Gigapixel-level 3D Reconstruction. Zhang, Jianing, et al. PAMI 2021.

• Multi-sensor large-scale dataset for multi-view 3D reconstruction. Oleg Voynov, etal. CVPR 2022.

Open Source

Project	Language	License
CMVS-PMVS	C++, CUDA	GPL
Colmap	C++, CUDA	BSD 3-Clause
Gipuma + Fusibile	C++, CUDA	GPL-3.0
MeshRoom(AliceVision)	C++, Python	MPL2
MVE	C++	BSD 3-Clause
OpenMVS	C++, CUDA(Optional)	AGPL3
MVS-Texturing	C++	BSD 3-Clause

Commercial Software

Software	Company
ContextCapture	Bentley Systems
DJI Terra	DJI
MetaShape	Agisoft
Pix4Dmapper	Pix4D
RealityCapture	Epic Games

License

MIT

Contribution

Welcome contribution!