NeurIPS 2024 SPOTLIGHT

3D Gaussian Splatting as Markov Chain Monte Carlo

Shakiba Kheradmand, Daniel Rebain, Gopal Sharma, Weiwei Sun, Yang-Che Tseng, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, Kwang Moo Yi

BibTeX

@article{kheradmand20243d,
  title={3D Gaussian Splatting as Markov Chain Monte Carlo},
  author={Kheradmand, Shakiba and Rebain, Daniel and Sharma, Gopal and Sun, Weiwei and Tseng, Jeff and Isack, Hossam and Kar, Abhishek and Tagliasacchi, Andrea and Yi, Kwang Moo},
  journal={arXiv preprint arXiv:2404.09591},
  year={2024}
}

How to Install

This project is built on top of the Original 3DGS code base and has been tested only on Ubuntu 20.04. If you encounter any issues, please refer to the Original 3DGS code base for installation instructions.

Installation Steps

Clone the Repository:

git clone --recursive https://github.com/ubc-vision/3dgs-mcmc.git
cd 3dgs-mcmc

Set Up the Conda Environment:

conda create -y -n 3dgs-mcmc-env python=3.8
conda activate 3dgs-mcmc-env

Install Dependencies:

pip install plyfile tqdm torch==1.13.1+cu117 torchvision==0.14.1+cu117 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu117
conda install cudatoolkit-dev=11.7 -c conda-forge

Install Submodules:

CUDA_HOME=PATH/TO/CONDA/envs/3dgs-mcmc-env/pkgs/cuda-toolkit/ pip install submodules/diff-gaussian-rasterization submodules/simple-knn/

Common Issues:

Access Error During Cloning: If you encounter an access error when cloning the repository, ensure you have your SSH key set up correctly. Alternatively, you can clone using HTTPS.
Running diff-gaussian-rasterization Fails: You may need to change the compiler options in the setup.py file to run both the original and this code. Update the setup.py with the following extra_compile_args:
```
extra_compile_args={"nvcc": ["-Xcompiler", "-fno-gnu-unique", "-I" + os.path.join(os.path.dirname(os.path.abspath(__file__)), "third_party/glm/")]}
```
Afterwards, you need to reinstall diff-gaussian-rasterization. This is mentioned in 3DGS-issue-#41.

By following these steps, you should be able to install the project and reproduce the results. If you encounter any issues, refer to the original 3DGS code base for further guidance.

How to run

Running code is similar to the Original 3DGS code base with the following differences:

You need to specify the maximum number of Gaussians that will be used. This is performed using --cap_max argument. The results in the paper uses the final number of Gaussians reached by the original 3DGS run for each shape.
You need to specify the scale regularizer coefficient. This is performed using --scale_reg argument. For all the experiments in the paper, we use 0.01.
You need to specify the opacity regularizer coefficient. This is performed using --opacity_reg argument. For Deep Blending dataset, we use 0.001. For all other experiments in the paper, we use 0.01.
You need to specify the noise learning rate. This is performed using --noise_lr argument. For all the experiments in the paper, we use 5e5.
You need to specify the initialization type. This is performed using --init_type argument. Options are random (to initialize randomly) or sfm (to initialize using a pointcloud).

How to Reproduce the Results in the Paper

python train.py --source_path PATH/TO/Shape --config configs/shape.json --eval

ubc-vision/3dgs-mcmc

** NeurIPS 2024 SPOTLIGHT **