Here is a collection of papers on the topic of 3D Image Editing, which encompasses research on Diffusion Models (DM), Neural Radiance Fields (NeRF), 3D Gaussian Splatting (3DGS) ,Large Language Models (LLMs), Vision Language Models (VLMs), Vision Foundation Models (VFMs), and various other methodologies.
[2024-5-26] Since the GPU resource is the most important indicator of whether a field is suitable to research, we decide to add GPU memory requirements for collected papers (Updating).
[2024-5-24] Ziyang Gong and Deblina Bhattacharjee curate this collection aiming to help researchers to know the development of this area. This repo also will be continually updating. If you have any question or advice, please feel free to contact us.
Ziyang Gong📧: gongzy23@sysu.mail2.edu.cn
Deblina Bhattacharjee📧: deblina.bhattacharjee@epfl.ch
Date |
Institute (first) | Paper Name |
Architecture |
Publication |
Code |
GPU Memory |
|---|---|---|---|---|---|---|
| 2024-04-08 | CAS | StylizedGS: Controllable Stylizationfor 3D Gaussian Splatting |
- | Arxiv | - | - |
| 2024-03-06 | CAS | DeferredGS: Decoupled and Editable Gaussian Splatting with Deferred Shading |
- | Arxiv | - | < 24GB |
Date |
Institute (first) | Paper Name |
Architecture |
Publication |
Code |
GPU Memory |
|---|---|---|---|---|---|---|
| 2024-02-22 | Tel Aviv | Consolidating Attention Features for Multi-view Image Editing |
- | Arxiv | Github | - |
| 2024-02-01 | UIUC | ViCA-NeRF: View-Consistency-Aware 3D Editing of Neural Radiance Fields |
- | NeurIPS 2023 | Github | - |
| 2023-09-18 | CUHK | RecolorNeRF: Layer Decomposed Radiance Fields for Efficient Color Editing of 3D Scenes |
- | ACM MM 2023 | Github | < 24GB |
| 2023-09-07 | SYSU | DreamEditor: Text-Driven 3D Scene Editing with Neural Fields |
- | SIGGRAPH Asia 2023 | Github | - |
| 2023-04-13 | Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields |
- | ICCV 2023 | - | - | |
| 2022-12-15 | CUHK | NeRF-Art: Text-Driven Neural Radiance Fields Stylization |
- | Arxiv | Github | - |
| 2022-11-22 | Samsung | SPIn-NeRF: Multiview Segmentation and Perceptual Inpainting with Neural Radiance Fields |
- | CVPR 2023 | Github | - |
Date |
Institute (first) | Paper Name |
Architecture |
Publication |
Code |
GPU Memory |
|---|---|---|---|---|---|---|
| 2024-03-25 | TUM | LightIt: Illumination Modeling and Control for Diffusion Models |
SD | CVPR 2024 | Github | - |
| 2023-12-05 | MIT | Alchemist: Parametric Control of Material Properties with Diffusion Models |
SD | Arxiv | Github | - |
| 2023-08-21 | SJTU | FocalDreamer: Text-Driven 3D Editing via Focal-Fusion Assembly |
SD | AAAI 2024 | Github | - |
| 2023-07-20 | AI2 | OBJECT 3DIT: Language-guided 3D-aware Image Editing |
DiT | NeurIPS 2023 | Page | - |
| 2022-08-02 | Prompt-to-Prompt Image Editing with Cross Attention Control |
SD | Arxiv | Github | - |
Date |
Institute (first) | Paper Name |
Architecture |
Publication |
Code |
GPU Memory |
|---|---|---|---|---|---|---|
| 2024-04-25 | SYSU+Tencent AI Lab | TIP-Editor: An Accurate 3D Editor Following Both Text-Prompts And Image-Prompts |
3DGS+SD | Arxiv | Github | - |
| 2024-04-17 | USTC | InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Prior |
3DGS+SD | Arxiv | Github | - |
| 2023-11-27 | Huawei | GaussianEditor: Editing 3D Gaussians Delicately with Text Instructions |
3DGS+SD+BLIP2+Dino+SAM | CVPR 2024 | Github | < 32GB |
Date |
Institute (first) | Paper Name |
Architecture |
Publication |
Code |
GPU Memory |
|---|---|---|---|---|---|---|
| 2024-04-23 | UCL | ProteusNeRF: Fast Lightweight NeRF Editing using 3D-Aware Image Context |
NeRF+DINO | I3D'24 | Github | - |
| 2024-03-27 | Tuebingen U | SIGNeRF: Scene Integrated Generation for Neural Radiance Fields |
NeRF+ControlNet | CVPR 2024 | Github | < 24GB |
| 2024-03-26 | EPFL | InNeRF360: Text-Guided 3D-Consistent Object Inpainting on 360◦ Neural Radiance Fields |
NeRF+Detection+SAM | CVPR 2024 | Github | - |
| 2024-03-21 | KAIST | ED-NERF: EFFICIENT TEXT-GUIDED EDITING OF 3D SCENE WITH LATENT SPACE NERF |
NeRF+SD | ICLR 2024 | Github | 8GB |
| 2024-03-08 | WHU | M²DNeRF: Multi-Modal Decomposition NeRF with 3D Feature Fields |
NeRF+Dino+CLIP-LSeg | Arxiv | - | < 24GB |
| 2024-01-02 | NYU | Image Sculpting: Precise Object Editing with 3D Geometry Control |
NeRF+SAM+DreamBooth+ControlNet+DDIM+SDXL | Arxiv | Github | < 24GB |
| 2023-12-26 | Apple | Efficient-NeRF2NeRF: Streamlining Text-Driven 3D Editing with Multiview Correspondence-Enhanced Diffusion Models |
IN2N | Arxiv | Github | 15GB |
| 2023-12-04 | IIE CAS | Customize your NeRF: Adaptive Source Driven 3D Scene Editing via Local-Global Iterative Training |
NeRF+LDM | Arxiv | Github | < 24GB |
| 2023-09-19 | Tel Aviv | Vox-E: Text-guided Voxel Editing of 3D Objects |
- | ICCV 2023 | Github | < 24GB |
| 2023-06-01 | UCB | Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions |
NeRF+SD | ICCV 2023 (Oral) | Github | 15GB |
| 2023-03-25 | ZJU | SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field |
NeRF+DINO-ViT | CVPR 2023 | Github | - |
| 2022-09-29 | DREAMFUSION: TEXT-TO-3D USING 2D DIFFUSION |
- | Arxiv | Github | - | |
| 2022-03-02 | CUHK | CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields |
NeRF+CLIP | Arxiv | Github | 17GB |