3D-Reconstruction-with-Deep-Learning-Methods

The focus of this list is on open-source projects hosted on Github.

Projects released on Github

TITLE KEYWORDS URL LICENSE Awesomeness
High Quality Monocular Depth Estimation via Transfer Learning TensorFlow, PyTorch https://github.com/ialhashim/DenseDepth https://arxiv.org/abs/1812.11941 GPL-3.0
Multi-view stereo image-based 3D reconstruction https://github.com/adahbingee/pais-mvs nn
Hybrid Ensemble Approach For 3D Object Reconstruction from Multi-View Monocular RGB images https://github.com/Ajithbalakrishnan/3D-Object-Reconstruction-from-Multi-View-Monocular-RGB-images nn
Deep 3D Semantic Scene Extrapolation hybrid CNN, GAN, TensorFlow https://github.com/AliAbbasi/Deep-3D-Semantic-Scene-Extrapolation http://user.ceng.metu.edu.tr/~ys/pubs/extrap-tvcj18.pdf nn
ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans TensorFlow https://github.com/angeladai/ScanComplete Apache-2.0
AtLoc: Attention Guided Camera Localization PyTorch https://github.com/BingCS/AtLoc https://arxiv.org/abs/1909.03557 BY-NC-SA 4.0
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation TensorFlow, cuDNN https://github.com/charlesq34/pointnet MIT License
PyTorch Implementation of DeepVO PyTorch, CNN https://github.com/ChiWeiHsiao/DeepVO-pytorch nn
Fully Convolutional Geometric Features: Fast and accurate 3D features for registration and correspondence. PyTorch https://github.com/chrischoy/FCGF MIT License
Morphing and Sampling Network for Dense Point Cloud Completion (AAAI2020) PyTorch https://github.com/Colin97/MSN-Point-Cloud-Completion Apache-2.0
Real-Time Self-Adaptive Deep Stereo TensorFlow https://github.com/CVLAB-Unibo/Real-time-self-adaptive-deep-stereo Apache-2.0
Geometry meets semantics for semi-supervised monocular depth estimation - ACCV 2018 TensorFlow https://github.com/CVLAB-Unibo/Semantic-Mono-Depth MIT License
BlenderProc: A procedural blender pipeline to generate images for deep learning Blender https://github.com/DLR-RM/BlenderProc GPL-3.0
SingleViewReconstruction: 3D Scene Reconstruction from a Single Viewport TensorFlow https://github.com/DLR-RM/SingleViewReconstruction MIT License
NNCAP — Neural Network Complex Approach to Photogrammetry https://github.com/Dok11/nn-dldm nn
Pytorch Implementation of Deeper Depth Prediction with Fully Convolutional Residual Networks PyTorch https://github.com/dontLoveBugs/FCRN_pytorch nn
Improved Adversarial Systems for 3D Object Generation and Reconstruction GAN https://github.com/EdwardSmith1884/3D-IWGAN MIT License
Deep Learning for Visual-Inertial Odometry PyTorch, CNN https://github.com/ElliotHYLee/Deep_Visual_Inertial_Odometry MIT License
Machine Vision List https://github.com/Ewenwan/MVision nn
Mesh R-CNN, an academic publication, presented at ICCV 2019 PyTorch, R-CNN https://github.com/facebookresearch/meshrcnn BSD-3-Clause License
PyTorch3d is FAIR's library of reusable components for deep learning with 3D data. PyTorch https://github.com/facebookresearch/pytorch3d BSD-3-Clause License
Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera PyTorch https://github.com/fangchangma/self-supervised-depth-completion MIT License
Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image PyTorch https://github.com/fangchangma/sparse-to-dense BSD License
Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image PyTorch https://github.com/fangchangma/sparse-to-dense.pytorch nn
PackNet-SfM: 3D Packing for Self-Supervised Monocular Depth Estimation PyTorch https://github.com/FangGet/PackNet-SFM-PyTorch GPL-3.0
InvSFM: Revealing Scenes by Inverting Structure from Motion Reconstructions [CVPR 2019] TensorFlow https://github.com/francescopittaluga/invsfm MIT License
Deep Monocular Visual Odometry using PyTorch (Experimental) PyTorch https://github.com/fshamshirdar/DeepVO nn
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation PyTorch https://github.com/fxia22/pointnet.pytorch MIT License
Pix2Depth - Depth Map Estimation from Monocular Image Keras https://github.com/gautam678/Pix2Depth GPL-3.0
3DRegNet: A Deep Neural Network for 3D Point Registration TensorFlow https://github.com/goncalo120/3DRegNet MIT License
Neural 3D Mesh Renderer – Single-Image 3D Reconstruction using Neural Renderer https://github.com/hiroharu-kato/mesh_reconstruction MIT License
Real-time Scalable Dense Surfel Mapping https://github.com/HKUST-Aerial-Robotics/DenseSurfelMapping nn
MVDepthNet: real-time multiview depth estimation neural network PyTorch https://github.com/HKUST-Aerial-Robotics/MVDepthNet nn
DeepMatchVO: Beyond Photometric Loss for Self-Supervised Ego-Motion Estimation https://github.com/hlzz/DeepMatchVO MIT License
MIRorR: Matchable Image Retrieval by Learning from Surface Reconstruction TensorFlow, CNN https://github.com/hlzz/mirror MIT License
Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstruction Caffe https://github.com/Huangying-Zhan/Depth-VO-Feat non-commercial
Deep Learning 3D vision papers papers, list, CN https://github.com/huayong/dl-vision-papers nn
Open3D PointNet implementation with PyTorch PyTorch, jupyter, Open3D https://github.com/intel-isl/Open3D-PointNet MIT License
Semantic-TSDF for Self-driving Static Scene Reconstruction PyTorch https://github.com/irsisyphus/semantic-tsdf MIT License
Weakly supervised 3D Reconstruction with Adversarial Constraint https://github.com/jgwak/McRecon MIT License
Using Deep learning Technique for Stereo vision and 3D reconstruction TensorFlow, CN https://github.com/jiafeng5513/EvisionNet nn
Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video PyTorch https://github.com/JiawangBian/SC-SfMLearner-Release GPL-3.0
Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundaries (official implementation) PyTorch https://github.com/JunjH/Revisiting_Single_Depth_Estimation nn
Visualization of Convolutional Neural Networks for Monocular Depth Estimation (official implementation) CNN, PyTorch https://github.com/JunjH/Visualizing-CNNs-for-monocular-depth-estimation MIT License
DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks PyTorch https://github.com/krrish94/DeepVO nn
DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction Tensorflow https://github.com/laughtervv/DISN nn
DeepTAM: Deep Tracking and Mapping https://github.com/lmb-freiburg/deeptam GPL-3.0
DeMoN: Depth and Motion Network Tensorflow https://github.com/lmb-freiburg/demon GPL-3.0
PyTorch implementation of CloudWalk's recent work DenseBody PyTorch https://github.com/Lotayou/densebody_pytorch GPL-3.0
Self-supervised learning for dense depth estimation in monocular endoscopy Tensorflow, Torch https://github.com/lppllppl920/EndoscopyDepthEstimation-Pytorch non-commercial
ContextDesc: Local Descriptor Augmentation with Cross-Modality Context Tensorflow https://github.com/lzx551402/contextdesc nn
GL3D (Geometric Learning with 3D Reconstruction): a large-scale database created for 3D reconstruction and geometry-related learning problems https://github.com/lzx551402/GL3D MIT License
Deeper Depth Prediction with Fully Convolutional Residual Networks (official implementation) Tensorflow https://github.com/MahmoudSelmy/DeeperDepthEstimation nn
Fine-Tuning Vgg16 For Depth Estimation Tensorflow https://github.com/MahmoudSelmy/DepthEstimationVGG nn
3D reconstruction with neural networks using Tensorflow. See link for Video https://github.com/micmelesse/3D-reconstruction-with-Neural-Networks nn
Learning Depth from Monocular Videos using Direct Methods PyTorch https://github.com/MightyChaos/LKVOLearner BSD-3-Clause
PointNetVLAD: Deep Point Cloud Based Retrieval for Large-Scale Place Recognition Tensorflow https://github.com/mikacuy/pointnetvlad MIT License
Attempting to estimate topography of a region from image data https://github.com/nbelakovski/topography_neural_net nn
DDRNet: Depth Map Denoising and Refinement for Consumer Depth Cameras Using Cascaded CNNs Tensorflow https://github.com/neycyanshi/DDRNet MIT License
Monocular depth estimation from a single image PyTorch https://github.com/nianticlabs/monodepth2 Copyright © Niantic, Inc. 2018. Patent Pending - non-commercial use only
3D-RelNet: Joint Object and Relation Network for 3D prediction Torch, jupyter https://github.com/nileshkulkarni/relative3d nn
PlaneRCNN detects and reconstructs piece-wise planar surfaces from a single RGB image Torch, RCNN https://github.com/NVlabs/planercnn Copyright (c) 2018 NVIDIA Corp. All Rights Reserved. This work is licensed under the Creative Commons Attribution NonCommercial ShareAlike 4.0 License.
OctoMap - An Efficient Probabilistic 3D Mapping Framework Based on Octrees. https://github.com/OctoMap/octomap University of Freiburg, Copyright (C) 2009-2014, octomap: New BSD License, octovis and related libraries: GPL
Unsupervised Monocular Depth Estimation neural network MonoDepth in PyTorch (Unofficial implementation) PyTorch https://github.com/OniroAI/MonoDepth-PyTorch nn
Learning to Sample: A learned sampling approach for point clouds https://github.com/orendv/learning_to_sample MIT License
DeepMVS: Learning Multi-View Stereopsis CNN, PyTorch https://github.com/phuang17/DeepMVS BSD 2-clause
DeepV2D: Video to Depth with Differentiable Structure from Motion Tensorflow https://github.com/princeton-vl/DeepV2D nn
High Quality Monocular Depth Estimation via Transfer Learning Tensorflow https://github.com/priya-dwivedi/Deep-Learning/tree/master/depth_estimation nn (GPL-3.0 ?)
Deep Single-View 3D Object Reconstruction with Visual Hull Embedding CNN, Tensorflow https://github.com/qweas120/PSVH-3d-reconstruction MIT License
ScanNet is an RGB-D video dataset containing 2.5 million views in more than 1500 scans, annotated with 3D camera poses,... https://github.com/ScanNet/ScanNet Can be used with the restriction to give credit and include original Copyright
Visual inspection of bridges is customarily used to identify and evaluate faults CNN https://github.com/Shaggyshak/CS543_project_Image-based-Localization-of-Bridge-Defects-with-AR-Visualization nn
Semantic 3D Occupancy Mapping through Efficient High Order CRFs CNN https://github.com/shichaoy/semantic_3d_mapping BSD-3-Clause
Factoring Shape, Pose, and Layout from the 2D Image of a 3D Scene https://github.com/shubhtuls/factored3d nn
Motion R-CNN codebase (old) RCNN https://github.com/simonmeister/old-motion-rcnn MIT License
Geometry-Aware Symmetric Domain Adaptation for Monocular Depth Estimation PyTorch https://github.com/sshan-zhao/GASDA nn
3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera https://github.com/StanfordVL/3DSceneGraph MIT License
Minkowski Engine is an auto-diff convolutional neural network library for high-dimensional sparse tensors PyTorch https://github.com/stanfordvl/MinkowskiEngine MIT License
Learning Single-View 3D Reconstruction with Limited Pose Supervision (Official implementation) Tensorflow https://github.com/stevenygd/3d-recon MIT License
VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera (Tensorflow version) Tensorflow https://github.com/timctho/VNect-tensorflow Apache-2.0
3D-LMNet: Latent Embedding Matching for Accurate and Diverse 3D Point Cloud Reconstruction from a Single Image https://github.com/val-iisc/3d-lmnet MIT License
Learning to Find Good Correspondences https://github.com/vcg-uvic/learned-correspondence-release For reserch and evaluation only. Commercial usage requires written approval
A Framework for the Volumetric Integration of Depth Images https://github.com/victorprad/InfiniTAM non-commercial
Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformation Tensorflow https://github.com/walsvid/Pixel2MeshPlusPlus BSD-3-Clause
Adversarial Semantic Scene Completion from a Single Depth Image (Official implementation) Tensorflow https://github.com/wangyida/gan-depth-semantic3d nn
SurfelWarp: Efficient Non-Volumetric Dynamic Reconstruction https://github.com/weigao95/surfelwarp BSD-3-Clause
PCN: Point Completion Network Tensorflow https://github.com/wentaoyuan/pcn MIT License
DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction https://github.com/Xharlie/DISN nn
Real-time motion from structure CNN https://github.com/yan99033/CNN-SVO nn
Dense 3D Object Reconstruction from a Single Depth View Tensorflow https://github.com/Yang7879/3D-RecGAN-extended MIT License
Semi-supervised monocular depth map prediction Tensorflow https://github.com/Yevkuzn/semodepth GPL-3.0
3DFeat-Net: Weakly Supervised Local 3D Features for Point Cloud Registration Tensorflow https://github.com/yewzijian/3DFeatNet MIT License
Estimated Depth Map Helps Image Classification: Depth estimation with neural network, and learning on RGBD images https://github.com/yihui-he/Estimated-Depth-Map-Helps-Image-Classification MIT License
Fit 3DMM to front and side face images simultaneously. https://github.com/Yinghao-Li/3DMM-fitting nn
The Perfect Match: 3D Point Cloud Matching with Smoothed Densities CNN, Tensorflow https://github.com/zgojcic/3DSmoothNet BSD-3-Clause
NeurVPS: Neural Vanishing Point Scanning via Conic Convolution Tenosorflow https://github.com/zhou13/neurvps MIT License
LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image (Torch implementation) Torch https://github.com/zouchuhang/LayoutNet MIT License
NeRF: Neural Radiance Fields https://github.com/bmild/nerf MIT License 10
Local Light Field Fusion at SIGGRAPH 2019 https://github.com/fyusion/llff GPL-3.0 10
neural-volumes-learning-dynamic-renderable-volumes-from-images https://research.fb.com/publications/neural-volumes-learning-dynamic-renderable-volumes-from-images/
https://github.com/facebookresearch/neuralvolumes
BY-NC 4.0
Learning Less is More - 6D Camera Localization via 3D Surface Regression https://github.com/vislearn/LessMore BSD-3-Clause
Local features https://github.com/vcg-uvic/lf-net-release
Pix2Vox https://github.com/hzxie/Pix2Vox
PlanarReconstruction: Single-Image Piece-wise Planar 3D Reconstruction via Associative Embedding pytorch https://github.com/svip-lab/PlanarReconstruction
Depth estimation with deep Neural networks https://medium.com/@omarbarakat1995/depth-estimation-with-deep-neural-networks-part-1-5fa6d2237d0d
https://medium.com/datadriveninvestor/depth-estimation-with-deep-neural-networks-part-2-81ee374888eb
https://github.com/MahmoudSelmy/DeeperDepthEstimation
https://github.com/MahmoudSelmy/DepthEstimationVGG/blob/master/README.md
High Quality Monocular Depth Estimation via Transfer Learning https://github.com/ialhashim/DenseDepth
https://arxiv.org/abs/1812.11941
D3Feat https://github.com/XuyangBai/D3Feat
Hierarchical Deep Stereo Matching on High Resolution Images pytorch https://github.com/gengshan-y/high-res-stereo MIT
Structure-Aware Residual Pyramid Network for Monocular Depth Estimation pytorch https://github.com/Xt-Chen/SARPN nn
Pytorch code to construct a 3D point cloud model from single RGB image. pytorch https://github.com/lkhphuc/pytorch-3d-point-cloud-generation nn
Depth estimation from RGB images using fully convolutional neural networks pytorch https://github.com/karoly-hars/DE_resnet_unet_hyb BSD-3-Clause
Single-Image Piece-wise Planar 3D Reconstruction via Associative Embedding torch, tensorflow https://github.com/svip-lab/PlanarReconstruction MIT
TriDepth: Triangular Patch-based Deep Depth Prediction PyTorch https://github.com/syinari0123/tridepth MIT
Depth Map Prediction from a Single Image using a Multi-Scale Deep Network torch https://github.com/imran3180/depth-map-prediction nn
Hybrid CNN for Single Image Depth Estimation torch https://github.com/karoly-hars/DE_resnet_unet_hyb BSD-3-Clause
MarrNet: 3D Shape Reconstruction via 2.5D Sketches torch https://github.com/jiajunwu/marrnet nn
Consistent Video Depth Estimation https://roxanneluo.github.io/Consistent-Video-Depth-Estimation/ nn
HF-Net: Robust Hierarchical Localization at Large Scale torch, tensorflow https://github.com/ethz-asl/hfnet MIT
Hierarchical Deep Stereo Matching on High Resolution Images pytorch https://github.com/gengshan-y/high-res-stereo MIT
Structure-Aware Residual Pyramid Network for Monocular Depth Estimation pytorch https://github.com/Xt-Chen/SARPN nn
Pytorch code to construct a 3D point cloud model from single RGB image. pytorch https://github.com/lkhphuc/pytorch-3d-point-cloud-generation nn
Depth estimation from RGB images using fully convolutional neural networks pytorch https://github.com/karoly-hars/DE_resnet_unet_hyb BSD-3-Clause
Single-Image Piece-wise Planar 3D Reconstruction via Associative Embedding torch, tensorflow https://github.com/svip-lab/PlanarReconstruction MIT
TriDepth: Triangular Patch-based Deep Depth Prediction PyTorch https://github.com/syinari0123/tridepth MIT
Depth Map Prediction from a Single Image using a Multi-Scale Deep Network torch https://github.com/imran3180/depth-map-prediction nn
Hybrid CNN for Single Image Depth Estimation torch https://github.com/karoly-hars/DE_resnet_unet_hyb BSD-3-Clause
MarrNet: 3D Shape Reconstruction via 2.5D Sketches torch https://github.com/jiajunwu/marrnet nn
Consistent Video Depth Estimation https://roxanneluo.github.io/Consistent-Video-Depth-Estimation/ nn
HF-Net: Robust Hierarchical Localization at Large Scale torch, tensorflow https://github.com/ethz-asl/hfnet MIT

Other Projects

TITLE KEYWORDS URL LICENSE
3D-Scene-GAN: Three-dimensional Scene Reconstruction with Generative Adversarial Networks paper https://openreview.net/forum?id=SkNEsmJwf
Google: Deep Learning Depth Prediction magazine article, GER https://www.digitalproduction.com/2019/05/27/google-deep-learning-depth-prediction/
SLAM and Deep Leraning for 3D Indoor Scene Understanding PhD thesis https://www.doc.ic.ac.uk/~ajd/Publications/McCormac-J-2019-PhD-Thesis.pdf
Dense 3D Object Reconstruction from a Single Depth View 3D-RecGAN++ https://arxiv.org/abs/1802.00411
Moving Camera, Moving People: A Deep Learning Approach to Depth Prediction https://ai.googleblog.com/2019/05/moving-camera-moving-people-deep.html
Depth Estimation from a Single RGB Image http://campar.in.tum.de/Chair/ProjectDepthPrediction
Deep Fundamental Matrix Estimation http://vladlen.info/papers/deep-fundamental.pdf
depth_estimation https://towardsdatascience.com/depth-estimation-on-camera-images-using-densenets-ac454caa893
3D-Machine-Learning List https://github.com/timzhang642/3D-Machine-Learning
DEEP LEARNING-BASED 3D OBJECT RECONSTRUCTION - A SURVEY - Image-based 3D Object Reconstruction:State-of-the-Art and Trends in the DeepLearning Era https://arxiv.org/pdf/1906.06543.pdf

I2-SDF: Intrinsic Indoor Scene Reconstruction and Editing via Raytracing in Neural SDFs (CVPR 2023) https://github.com/jingsenzhu/i2-sdf MIT

https://github.com/lioryariv/idr

https://github.com/autonomousvision/differentiable_volumetric_rendering

https://github.com/Dok11/surface-match-dataset

Image-based 3D Object Reconstruction:State-of-the-Art and Trends in the DeepLearning Era https://arxiv.org/pdf/1906.06543v3.pdf

Dense 3D Object Reconstructionfrom a Single Depth View https://arxiv.org/pdf/1802.00411v2.pdf

https://dagshub.com/OperationSavta/SavtaDepth https://colab.research.google.com/drive/1XU4DgQ217_hUMU1dllppeQNw3pTRlHy1?usp=sharing https://huggingface.co/spaces/kingabzpro/savtadepth MIT License

https://github.com/gradslam/gradslam pyTorch

https://github.com/ventusff/neurecon

https://github.com/theICTlab/3DUNDERWORLD-SLS-GPU_CPU