Generative Densification: Learning to Densify Gaussians
for High-Fidelity Generalizable 3D Reconstruction

Seungtae Nam* · Xiangyu Sun* · Gyeongjin Kang · Younggeun Lee · Seungjun Oh · Eunbyung Park

Paper | Project Page | Checkpoints


Our method selectively densifies coarse Gaussians generated by generalized feed-forward models.

Installation

1. Install Docker and NVIDIA Container Toolkit

please follow the official document for installation.
if you already installed both of them, please skip this part.

2. Clone

git clone https://github.com/stnamjef/GenerativeDensification.git --recursive

3. Create Environment

please prepare the datasets first following the instructions.
then run below at /your/path/to/GenerativeDensification (notice the dot at the end).

docker build -t generative_densification:0.0.1 .
docker run -it -v $(pwd):/workspace --ipc host --gpus all generative_densification:0.0.1

4. Install diff-gaussian-rasterization

run below at /workspace inside the docker container.

pip3 install ./third_party/diff-gaussian-rasterization

Datasets

Our object-level model is trained on Gobjaverse training set, provided by LaRa.
We do cross-dataset generalization on GSO and Co3D dataset. You can download our preprocessed Co3D dataset here.
Note:
- The Gobjaverse dataset requires 1.4TB of storage.
- We assume the datasets are in the ./GenerativeDensification/dataset.

GenerativeDensification
├── dataLoader
├── dataset
│   ├── gobjaverse
│   │   ├── gobjaverse_part_01.h5
│   │   ...
│   │
│   ├── google_scanned_objects
│   │   ├── 2_of_Jenga_Classic_Game
│   │   ...
│   ...
├── lightning
...

Training

You can enable residual learning by setting model.enable_residual_attribute=True.

python train_lightning.py \
  train_dataset.data_root=./dataset/gobjaverse/gobjaverse.h5 \
  test_dataset.data_root=./dataset/gobjaverse/gobjaverse.h5 \
  model.enable_residual_attribute=False

Evaluation

We provide two checkpoints (w/ residual learning and w/o it) for our object-level models.
Note:
- The checkpoint 'epoch=49.ckpt' corresponds to 'Ours' model in the paper.
- The checkpoint 'epoch=49_residual.ckpt' corresponds to 'Ours (w/ residual)' model in the paper.

python eval_all.py

Acknowledgements

Our work is built upon the following projects. We thank all the authors for making their amazing works publicly available.

Citation

@article{GenerativeDensification,
  title={Generative Densification: Learning to Densify Gaussians for High-Fidelity Generalizable 3D Reconstruction}, 
  author={Nam, Seungtae and Sun, Xiangyu and Kang, Gyeongjin and Lee, Younggeun and Oh, Seungjun and Park, Eunbyung},
  journal={arXiv preprint arXiv:2412.06234},
  year={2024}
}