This project merges DUSt3R's capabilities in camera parameter estimation and point cloud creation with the 3D scene representation efficiency of Gaussian Splatting. The goal is to simplify the process of 3D scene reconstruction and visualization from images without requiring pre-set camera information or specific viewpoint data.
To begin, clone the repository and initialize its submodules:
git clone git@github.com:nerlfield/wild-gaussian-splatting.git --recursive
cd wild-gaussian-splatting
git pull --recurse-submodules
git submodule update --init --recursive
conda create -n wildgaussians python=3.11 cmake=3.14.0 -y
conda activate wildgaussians
With the environment activated, run the provided setup script and pass your CUDA version as an argument. This script installs necessary dependencies tailored to your CUDA version:
./setup_environment.sh <cuda_version>
Replace <cuda_version> with the version of CUDA you intend to use (e.g., 10.2, 11.1, 12.1). I used 12.1
.
This repository contains two notebooks that showcase fitting gaussians over scene from the wild:
./notebooks/00_dust3r_inference.ipynb
- Runs DUSt3R on a folder of images, saving camera parameters and point clouds in COLMAP format. Note: DUSt3R's memory usage increases quadratically with the number of images. On an L4 instance, it can process up to 32 images of 512x384 size../notebooks/01_gaussian_splatting_fitting.ipynb
- Reads DUSt3R results and applies Gaussian splatting. Modifications include the addition of rendering camera trajectory generation and bug fixes.
To launch Jupyter Lab, use:
jupyter lab --no-browser --ip 0.0.0.0 --port 4546 --allow-root --notebook-dir=.