/NeuralPoissonSolver

PyTorch implementation of Neural Poisson Solver.

Primary LanguagePython

[ECCV 2024] Neural Poisson Solver: A Universal and Continuous Framework for Natural Signal Blending

PyTorch implementation of Neural Poisson Solver.

Pipeline

pipeline

Setup

We provide a conda environment setup file including all of the above dependencies. Create the conda environment Neural Poisson Solver by running:

conda create -n neural-poisson-solver python=3.8
conda activate neural-poisson-solver
pip install -r requirements.txt

Running

2D scene

For 2D scene blending tasks, we employ DINER as the backbone network.

Data preparation

You need to prepare the following data and place them in the data/2d/ folder. The directory structure should look like this:

data/2d/
├── scene_1/
│   ├── src.pth          # Source scene's INR model
│   ├── tgt.pth          # Target scene's INR model
│   ├── roi.png          # Blending region
│   ├── cfg.npy          # Blending center coordinates
└── ...

Scene blending

export PYTHONPATH=$(pwd)
CUDA_VISIBLE_DEVICES=0 python src/blending/blend_2d.py --save_dir results/2d/scene_1/ --root_dir data/2d/scene_1/ --use_numpy False

3D scene

For 3D scene blending tasks, we employ NeRF as the backbone network.

Data preparation

You need to prepare the following data and place them in the data/3d/ folder. The directory structure should look like this:

data/3d/
├── scene_1/
│   ├── src.tar          # Source scene's NeRF model
│   ├── tgt.tar          # Target scene's NeRF model
│   ├── roi.pt           # Blending region
│   ├── cam.json         # Camera parameters
└── ...

Scene blending

export PYTHONPATH=$(pwd)
sh src/models/nerf/install_ext.sh
CUDA_VISIBLE_DEVICES=0 python src/blending/blend_3d.py --save_dir results/3d/scene_1/ --root_dir data/3d/scene_1/

Citation

@InProceedings{10.1007/978-3-031-72989-8_15,
	author="Wu, Delong
	and Zhu, Hao
	and Zhang, Qi
	and Li, You
	and Ma, Zhan
	and Cao, Xun",
	editor="Leonardis, Ale{\v{s}}
	and Ricci, Elisa
	and Roth, Stefan
	and Russakovsky, Olga
	and Sattler, Torsten
	and Varol, G{\"u}l",
	title="Neural Poisson Solver: A Universal and Continuous Framework for Natural Signal Blending",
	booktitle="Computer Vision -- ECCV 2024",
	year="2025",
	publisher="Springer Nature Switzerland",
	address="Cham",
	pages="259--275",
}