This repository collects the studies on 3D generation, including both 3D shape generation and 3D-aware image generation. Different from 3D reconstruction, which focuses on per-instance recovery (i.e., the data already exists in the real world), 3D generation targets learning the real distribution and hence allows sampling new data.
Overall, the paper collection is organized as follows. If you find some work is missing, feel free to raise an issue or create a pull request. We appreciate contributions in any form.
We categorize the studies on 3D shape generation according to the representation used.
- Learning Representations and Generative Models for 3D Point Clouds
ICML 2018 / Code - Multiresolution Tree Networks for 3D Point Cloud Processing
ECCV 2018 / Code / Project Page - 3D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutions
ICCV 2019 / Code - Point Cloud GAN
ICLR 2019 / Code - Learning Localized Generative Models for 3D Point Clouds via Graph Convolution
ICLR 2019 / Code - PointFlow : 3D Point Cloud Generation with Continuous Normalizing Flows
ICCV 2019 / Code - Spectral-GANs for High-Resolution 3D Point-Cloud Generation
IROS 2020 / Code - Progressive Point Cloud Deconvolution Generation Network
ECCV 2020 / Code - A Progressive Conditional Generative Adversarial Network for Generating Dense and Colored 3D Point Clouds
3DV 2020 / Code - Adversarial Autoencoders for Generating 3D Point Clouds
ICLR 2020 / Code - Learning Gradient Fields for Shape Generation
ECCV 2020 / Code / Project Page - SoftFlow: Probabilistic Framework for Normalizing Flow on Manifolds
NeurIPS 2020 / Code - Discrete Point Flow Networks for Efficient Point Cloud Generation
ECCV 2020 / Code - Pointgrow: Autoregressively Learned Point Cloud Generation with Self-Attention
WACV 2020 / Code / Project Page - MRGAN: MultiRooted 3D Shape Generation with Unsupervised Part Disentanglement
ICCVW 2021 - SP-GAN: Sphere-Guided 3D Shape Generation and Manipulation
SIGGRAPH 2021 / Code - Generative PointNet: Deep Energy-Based Learning on Unordered Point Sets for
3D Generation, Reconstruction and Classification
CVPR 2021 / Code / Project Page
- 3d Shapenets: A Deep Representation for Volumetric Shapes
CVPR 2015 / Code / Project Page - Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
NeurIPS 2016 / Code / Project Page - Generative and Discriminative Voxel Modeling with Convolutional Neural Networks
arXiv 2016 / Code - Octree Generating Networks: Efficient Convolutional Architectures for High-Resolution 3D Outputs
ICCV 2017 / Code - SAGNet: Structure-aware Generative Network for 3D-Shape Modeling
SIGGRAPH 2019 / Code / Project Page - Generalized Autoencoder for Volumetric Shape Generation
CVPRW 2020 / Code - PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes
CVPR 2020 / Code - Learning Part Generation and Assembly for Structure-Aware Shape Synthesis
AAAI 2020 - Generative VoxelNet: Learning Energy-Based Models for 3D Shape Synthesis and Analysis
TPAMI 2020 / Code / Project Page - DLGAN: Depth-Preserving Latent Generative Adversarial Network for 3D Reconstruction
TMM 2020 - DECOR-GAN: 3D Shape Detailization by Conditional Refinement
CVPR 2021 / Code - Octree Transformer: Autoregressive 3D Shape Generation on Hierarchically Structured Sequences
arXiv 2021
- SDM-Net: Deep Generative Network for Structured Deformable Mesh
SIGGRAPH Asia 2019 / Project Page - PolyGen: An Autoregressive Generative Model of 3D Meshes
ICML 2020 / Code - TM-NET: Deep Generative Networks for Textured Meshes
TOG 2021 / Code / Project Page
- Learning Implicit Fields for Generative Shape Modeling
CVPR 2019 / Code / Project Page - Adversarial Generation of Continuous Implicit Shape Representations
arXiv 2020 / Code - DualSDF: Semantic Shape Manipulation using a Two-Level Representation
CVPR 2020 / Code / Project Page - SurfGen: Adversarial 3D Shape Synthesis with Explicit Surface Discriminators
ICCV 2021 - 3D Shape Generation with Grid-Based Implicit Functions
CVPR 2021 - gDNA: Towards Generative Detailed Neural Avatars
CVPR 2022 / Code / Project Page - Deformed Implicit Field: Modeling 3D shapes with Learned Dense Correspondence
CVPR 2021 / Code
- Multi-Chart Generative Surface Modeling
SIGGRAPH Asia 2018 / Code
- Physically-Aware Generative Network for 3D Shape Modeling
CVPR 2021
- Coupling Explicit and Implicit Surface Representations for Generative 3D Modeling
ECCV 2020 - Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis
NeurIPS 2021 / Project Page
- ShapeAssembly: Learning to Generate Programs for 3D Shape Structure Synthesis
SIGGRAPH Asia 2020 / Code / Project Page
We categorize the studies on 3D-aware image generation according to the representation used.
- Points2Pix: 3D Point-Cloud to Image Translation using conditional Generative Adversarial Networks
arXiv 2019
- Visual Object Networks: Image Generation with Disentangled 3D Representation
NeurIPS 2018 / Code / Project Page - HoloGAN: Unsupervised Learning of 3D representations from Natural Images
ICCV 2019 / Code - BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Images
NeurIPS 2020 / Code - Towards a Neural Graphics Pipeline for Controllable Image Generation
Computer Graphics Forum 2021 / Project Page
- Generative Image Modeling using Style and Structure Adversarial Networks
ECCV 2016 / Code - Geometric Image Synthesis
ACCV 2018 - RGBD-GAN: Unsupervised 3D Representation Learning From Natural Image Datasets via RGBD Image Synthesis
ICLR 2020 / Code - 3D-Aware Indoor Scene Synthesis with Depth Priors
arXiv 2022 / Code / Project Page
- GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis
NeurIPS 2020 / Code / Project Page - GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields
CVPR 2021 / Code / Project Page - pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis
CVPR 2021 / Code / Project Page - Unconstrained Scene Generation with Locally Conditioned Radiance Fields
ICCV 2021 / Code / Project Page - A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis
NeurIPS 2021 / Code / Project Page - Campari: Camera-aware Decomposed Generative Neural Radiance Fields
3DV 2021 / Code - CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesis
arXiv 2021 / Code - GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worlds
ICCV 2021 / Code / Project Page - Generative Occupancy Fields for 3D Surface-Aware Image Synthesis
NeurIPS 2021 / Code / Project Page - 3D-Aware Semantic-Guided Generative Model for Human Synthesis
arXiv 2021 - FENeRF: Face Editing in Neural Radiance Fields
CVPR 2022 - StyleNeRF: A Style-Based 3D-Aware Generator for High-resolution Image Synthesis
ICLR 2022 / Code / Project Page - StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation
CVPR 2022 / Code / Project Page - GRAM: Generative Radiance Manifolds for 3D-Aware Image Generation
CVPR 2022 / Project Page - A Generative Model for 3D Face Synthesis with HDRI Relighting
arXiv 2022 - Pix2NeRF: Unsupervised Conditional π-GAN for Single Image to Neural Radiance Fields Translation
CVPR 2022 - 3D-GIF: 3D-Controllable Object Generation via Implicit Factorized Representations
arXiv 2022 - Generating Videos with Dynamics-aware Implicit Generative Adversarial Networks
ICLR 2022 / Code / Project Page
- 3D-aware Image Synthesis via Learning Structural and Textural Representations
CVPR 2022 / Code / Project Page - Efficient Geometry-aware 3D Generative Adversarial Networks
CVPR 2022 / Code / Project Page
Besides explicitly learning a 3D generative model, there are also some attempts working on the 3D controllability of 2D models.
- Interpreting the latent space of gans for semantic face editing
CVPR 2020 / Code / Project Page - StyleRig: Rigging StyleGAN for 3D Control over Portrait Images
CVPR 2020 / Project Page - Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning
CVPR 2020 / Code - Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesis
CVPR 2020 / Code - DiscoFaceGAN: Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning
CVPR 2020 / Code - FreeStyleGAN: Free-view Editable Portrait Rendering with the Camera Manifold
SIGGRAPH Asia 2021 / Code / Project Page - Do 2D GANs Know 3D Shape? Unsupervised 3D Shape Reconstruction from 2D Image GANs
ICLR 2021 / Code / Project Page - Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis
IJCV 2021 / Code / Project Page - Cross-Domain and Disentangled Face Manipulation with 3D Guidance
TVCG 2021 / Code / Project Page - Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering
ICLR 2021 / Project Page - Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGAN
TOG 2021 / Code / Project Page - SofGAN: A Portrait Image Generator with Dynamic Styling
TOG 2021 / Code / Project Page - MOST-GAN: 3D Morphable StyleGAN for Disentangled Face Image Manipulation
AAAI 2022