/Awesome-Object-Compositional-INR

:book: A collection of object-compositional modeling by implicit neural representation.

Awesome Object Compositional Implicit Neural Representation (INR) Awesome

This repo contains a curative list of object-compositional modeling by implicit neural representation.

Motivation of this collection

Our inspiration was sparked by the rapidly growing trend of implicit neural representations (INR), such as NeRF, which have displayed remarkable capabilities in scene representation. Given that our environment comprises diverse objects, it raises an intriguing question: How can we comprehensively represent the basic constituents of a scene, the objects, while also promoting compositional capacity? Thus, our project involves gathering research papers that provide possible solutions to this problem, and we've labeled this collection as 'Object-Compositional Implicit Neural Representation'.

🔆 This project is still ongoing, pull requests are welcomed!!

If you have any suggestions (missing papers, new papers or typos), please feel free to edit and pull a request. Just letting me know the title of papers can also be a big contribution to me. You can do this by opening issues.

⭐️ If you find this repo useful, welcome to star it.

Papers

Learning from a single scene

  • Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering. (ICCV 2021) Paper | Project Page | Code
  • In-Place Scene Labelling and Understanding with Implicit Scene Representation. (ICCV 2021) Paper | Project Page | Code
  • Object-compositional Neural Implicit Surfaces. (ECCV 2022) Paper | Project Page | Code
  • Neural Feature Fusion Fields (N3F): 3D Distillation of Self-Supervised 2D Image Representations (3DV 2022) Paper | Project Page | Code
  • Decomposing nerf for editing via feature field distillation. (NeurIPS 2022) Paper | Project Page | Code
  • Feature-realistic neural fusion for real-time, open set scene understanding. (ICRA 2023) Paper | Project Page
  • Dm-nerf: 3d scene geometry decomposition and manipulation from 2d images. (ICLR 2023) Paper | Code
  • NeRF-SOS: Any-View Self-supervised Object Segmentation on Complex Scenes. (ICLR 2023) Paper | Project Page | Code
  • vMAP: Vectorised Object Mapping for Neural Field SLAM. (CVPR 2023) Paper | Project Page | Code
  • Nerflets: Local Radiance Fields for Efficient Structure-Aware 3D Scene Representation from 2D Supervision. (CVPR 2023) Paper | Project Page
  • Panoptic lifting for 3d scene understanding with neural fields. (CVPR 2023) Paper | Project Page | Code
  • Panoptic Compositional Feature Field for Editable Scene Rendering With Network-Inferred Labels via Metric Learning. (CVPR 2023) Paper
  • Automated 3D Object Discovery and Reconstruction. (CVPR 2023) Paper | Project Page | Code
  • Rico: Regularizing the unobservable for indoor compositional reconstruction. (ICCV 2023) Paper | Code
  • AssetField: Assets Mining and Reconfiguration in Ground Feature Plane Representation. (ICCV 2023) Paper | Project Page
  • Learning Unified Decompositional and Compositional NeRF for Editable Novel View Synthesis. (ICCV 2023) Paper | Project Page

Learning from dynamic scene

  • Neural scene graphs for dynamic scenes. (CVPR 2021) Paper | Project Page | Code
  • NeuralDiff: Segmenting 3D objects that move in egocentric videos. (3DV 2021) Paper | Project Page | Code
  • Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representation. (CVPR 2022) Paper | Project Page
  • D2nerf: self-supervised decoupling of dynamic and static objects from a monocular video. (NeurIPS 2022) Paper | Project Page | Code
  • SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videos. (NeurIPS 2022) Paper | Project Page | Code
  • Conditional Object-Centric Learning from Video. (ICLR 2022) Paper | Project Page | Code
  • NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields. (TVCG 2023) Paper | Project Page | Code
  • MARS: An Instance-aware, Modular and Realistic Simulator for Autonomous Driving (CICAI 2023) Paper | Project Page | Code

Learning from dataset

Object-Compositional Scene Generation

  • GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields. (CVPR 2021) Paper | Project Page | Code
  • Unconstrained scene generation with locally conditioned radiance fields. (ICCV 2021) Paper | Project Page | Code
  • DisCoScene: Spatially Disentangled Generative Radiance Fields for Controllable 3D-aware Scene Synthesis. (CVPR 2023) Paper | Project Page | Code
  • UrbanGIRAFFE: Representing Urban Scenes as Compositional Generative Neural Feature Fields Paper | Project Page
  • CompoNeRF: Text-guided Multi-object Compositional NeRF with Editable 3D Scene Layout Paper
  • Set-the-Scene: Global-Local Training for Generating Controllable NeRF Scenes Paper| Project Page | Code
  • Compositional 3D Scene Generation using Locally Conditioned Diffusion Paper | Project Page
  • CC3D: Layout-Conditioned Generation of Compositional 3D Scenes (ICCV 2023) Paper | Project Page | Code

Related Repository.

To capture or understand the surrounding world from an object-compositional perspective, there are some other awesome collections that could help you dive into this area.