/awesome-3D-generation

A curated list of awesome 3d generation papers

Awesome 3D Generation

News: We provide a survey, Deep Generative Models on 3D Representations: A Survey, to help the community track the evolution of this field.

Deep Generative Models on 3D Representations: A Survey
Zifan Shi*, Sida Peng*, Yinghao Xu*, Yiyi Liao, Yujun Shen
https://arxiv.org/abs/2210.15663
(* denotes equal contribution)

Overview

This repository collects the studies on 3D generation, including both 3D shape generation and 3D-aware image generation. Different from 3D reconstruction, which focuses on per-instance recovery (i.e., the data already exists in the real world), 3D generation targets learning the real distribution and hence allows sampling new data.

Overall, the paper collection is organized as follows. If you find some work is missing, feel free to raise an issue or create a pull request. We appreciate contributions in any form.

3D Shape Generation

We categorize the studies on 3D shape generation according to the representation used.

Point Cloud

  • Learning Representations and Generative Models for 3D Point Clouds
    ICML 2018 / Code
  • Multiresolution Tree Networks for 3D Point Cloud Processing
    ECCV 2018 / Code / Project Page
  • 3D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutions
    ICCV 2019 / Code
  • Point Cloud GAN
    ICLR 2019 / Code
  • Learning Localized Generative Models for 3D Point Clouds via Graph Convolution
    ICLR 2019 / Code
  • PointFlow : 3D Point Cloud Generation with Continuous Normalizing Flows
    ICCV 2019 / Code
  • Spectral-GANs for High-Resolution 3D Point-Cloud Generation
    IROS 2020 / Code
  • Progressive Point Cloud Deconvolution Generation Network
    ECCV 2020 / Code
  • A Progressive Conditional Generative Adversarial Network for Generating Dense and Colored 3D Point Clouds
    3DV 2020 / Code
  • Adversarial Autoencoders for Generating 3D Point Clouds
    ICLR 2020 / Code
  • Learning Gradient Fields for Shape Generation
    ECCV 2020 / Code / Project Page
  • SoftFlow: Probabilistic Framework for Normalizing Flow on Manifolds
    NeurIPS 2020 / Code
  • Discrete Point Flow Networks for Efficient Point Cloud Generation
    ECCV 2020 / Code
  • Pointgrow: Autoregressively Learned Point Cloud Generation with Self-Attention
    WACV 2020 / Code / Project Page
  • MRGAN: MultiRooted 3D Shape Generation with Unsupervised Part Disentanglement
    ICCVW 2021
  • SP-GAN: Sphere-Guided 3D Shape Generation and Manipulation
    SIGGRAPH 2021 / Code
  • Generative PointNet: Deep Energy-Based Learning on Unordered Point Sets for 3D Generation, Reconstruction and Classification
    CVPR 2021 / Code / Project Page
  • Diffusion Probabilistic Models for 3D Point Cloud Generation
    CVPR 2021 / Code
  • 3D Shape Generation and Completion through Point-Voxel Diffusion
    ICCV 2021 / Code / Project Page
  • ManiFlow: Implicitly Representing Manifolds with Normalizing Flows
    3DV 2022
  • LION: Latent Point Diffusion Models for 3D Shape Generation
    NeurIPS 2022 / Code / Project Page

Voxel

  • Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
    NeurIPS 2016 / Code / Project Page
  • Generative and Discriminative Voxel Modeling with Convolutional Neural Networks
    arXiv 2016 / Code
  • SAGNet: Structure-aware Generative Network for 3D-Shape Modeling
    SIGGRAPH 2019 / Code / Project Page
  • Generalized Autoencoder for Volumetric Shape Generation
    CVPRW 2020 / Code
  • PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes
    CVPR 2020 / Code
  • Learning Part Generation and Assembly for Structure-Aware Shape Synthesis
    AAAI 2020
  • Generative VoxelNet: Learning Energy-Based Models for 3D Shape Synthesis and Analysis
    TPAMI 2020 / Code / Project Page
  • Octree Transformer: Autoregressive 3D Shape Generation on Hierarchically Structured Sequences
    arXiv 2021
  • AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generation
    CVPR 2022 / Project Page / Code

Mesh

Neural Field

  • Learning Implicit Fields for Generative Shape Modeling
    CVPR 2019 / Code / Project Page
  • Adversarial Generation of Continuous Implicit Shape Representations
    EG 2020 / Code
  • DualSDF: Semantic Shape Manipulation using a Two-Level Representation
    CVPR 2020 / Code / Project Page
  • Physically-Aware Generative Network for 3D Shape Modeling
    CVPR 2021
  • SurfGen: Adversarial 3D Shape Synthesis with Explicit Surface Discriminators
    ICCV 2021
  • 3D Shape Generation with Grid-Based Implicit Functions
    CVPR 2021 / Code
  • Deformed Implicit Field: Modeling 3D shapes with Learned Dense Correspondence
    CVPR 2021 / Code
  • gDNA: Towards Generative Detailed Neural Avatars
    CVPR 2022 / Code / Project Page
  • ShapeCrafter: A Recursive Text-Conditioned 3D Shape Generation Model
    arXiv 2022
  • Learning to Generate 3D Shapes from a Single Example
    arXiv 2022 / Code / Project Page
  • 3D-LDM: Neural Implicit 3D Shape Generation with Latent Diffusion Models
    arXiv 2022

Program

3D-aware Image Generation

We categorize the studies on 3D-aware image generation according to the representation used.

Voxel

Depth

  • Generative Image Modeling using Style and Structure Adversarial Networks
    ECCV 2016 / Code
  • RGBD-GAN: Unsupervised 3D Representation Learning From Natural Image Datasets via RGBD Image Synthesis
    ICLR 2020 / Code
  • 3D-Aware Indoor Scene Synthesis with Depth Priors
    ECCV 2022 / Code / Project Page

Neural Field

Hybrid Representation

3D Control of 2D Generative Models

Besides explicitly learning a 3D generative model, there are also some attempts working on the 3D controllability of 2D models.