An official implementation for Splatfacto-W.
Please update your nerfstudio to the latest development version.
git clone https://github.com/nerfstudio-project/nerfstudio.git
cd nerfstudio
pip install --upgrade pip setuptools
pip install -e .
- Support rendering videos for selected camera index
- Support exporting PLY files for selected index
- (Probably) Merge splatfacto-w-light into Nerfstudio main branch
Ensure that nerfstudio has been installed according to the instructions. Clone or fork this repository and run the commands:
conda activate nerfstudio
cd splatfacto-w/
pip install -e .
ns-install-cli
You can download the phototourism dataset from running.
ns-download-data phototourism --capture-name <capture_name>
To train with it, download the train/test tsv file from the bottom of nerf-w and put it under the data folder (or copy them from ./splatfacto-w/dataset_split). For instance, for Brandenburg Gate the path would be your-data-folder/brandenburg_gate/brandenburg.tsv. You should have the following structure in your data folder:
|---brandenburg_gate
| |---dense
| | |---images
| | |---sparse
| | |---stereo
| |---brandenburg.tsv
Then, run the command:
ns-train splatfacto-w --data [PATH] nerf-w-data-parser-config --data_name [trevi,sacre,brandenburg]
If you want to train datasets without nerf-w's train/test split or your own datasets, we provided a light-weight version of the method for general cases. To train with it, you can run the following command
ns-train splatfacto-w-light [OPTIONS] --data [PATH] [dataparser]You can add these options to enhance the training process:
-
--pipeline.model.enable_bg_model True- Enables background modeling
- Useful for scenes with distinct foreground and background
-
--pipeline.model.enable_alpha_loss True- Enables alpha loss for punishing gaussians from occupying sky areas.
- Should be used with background modeling
-
--pipeline.model.enable_robust_mask True- Improves handling of transient objects (things that appear/disappear between views)
For phototourism, the dataparser should be colmap and you need to change the colmap path through the CLI because phototourism dataparser does not load 3D points.
We provide a temporary script export_script.py for exporting PLY files. Notice that the background model won't be exported since it is not supported by other PLY viewers.
The basic command structure is as follows:
python export_script.py --load_config [config_file] --output_dir [output_directory] --camera_idx [camera_index]
You can see the current camera index in the CLI as you click each image in Viser.
To render videos for selected camera index, add the --camera_idx [camera_index] parameter to the end of your ns-render command.