Project Page | Paper | ArXiv
Reimplementation of InstantSplat
Sparsed-view Dynamic 3D Gaussians in the Wild
Juan Atehortúa 1,
Alice Yu 1
1 Massachusetts Institute of Technology
TBD, 2024
ate@mit.edu
# Install this repo (pytorch)
git clone --recursive https://github.com/Atehortuajf/Dynamic3DGaussians.git
conda env create --file environment.yml
conda activate dynamic_gaussians
# Install Gaussian Rasterizer
cd diff-gaussian-rasterization-w-depth
python setup.py install
pip install .
# Optional but highly recommended, compile curope stuff for dust3r
cd dust3r/croco/models/curope/
python setup.py build_ext --inplace
cd ../../../
mkdir -p checkpoints/
wget https://download.europe.naverlabs.com/ComputerVision/DUSt3R/DUSt3R_ViTLarge_BaseDecoder_512_dpt.pth -P checkpoints/We provide a hydra config tool config/train.yaml to specify the folder to the training data and other hyperparameters. In the folder, the program expects multiple .mp4 videos. Once the configuration is set, simply run python train.py to run the training loop.
A config file named config/visualize.yaml exists to specify what experiment and sequence to visualize.
The code is essentially a reimplementation of InstantSplat on top of the Dynamic 3D Gaussian code, with the addition of hydra to make it easier to experiment.
This code uses uses the OpenCV camera co-ordinate system (same as COLMAP). This is different to the blender / standard NeRF camera coordinate system. The conversion code between the two can be found here
tbd