This is the repo for the implementation of F2-NeRF: Fast Neural Radiance Field Training with Free Camera Trajectories.
Project page | Paper | Data
The development of this project is primarily based on LibTorch.
For Debian based Linux distributions:
sudo apt install zlib1g-dev
For Arch based Linux distributions:
sudo pacman -S zlib
git clone --recursive https://github.com/Totoro97/f2-nerf.git
cd f2-nerf
We take torch-1.13.1+cu117
for example.
cd External
wget https://download.pytorch.org/libtorch/cu117/libtorch-cxx11-abi-shared-with-deps-1.13.1%2Bcu117.zip
unzip ./libtorch-cxx11-abi-shared-with-deps-1.13.1%2Bcu117.zip
The lowest G++ version I have tested is 7.5.0.
cd ..
cmake . -B build
cmake --build build --target main --config RelWithDebInfo -j
Here is an example command to train F2-NeRF:
python scripts/run.py --config-name=wanjinyou dataset_name=example case_name=ngp_fox mode=train
Simply run:
python scripts/run.py --config-name=wanjinyou dataset_name=example case_name=ngp_fox mode=test is_continue=true
We provide a script to generate render path (by interpolating the input camera poses). For example, for the fox data, run:
python scripts/inter_poses.py --data_dir ./data/example/ngp_fox --key_poses 5,10,15,20,25,30,35,40,45,49 --n_out_poses 200
The file poses_render.npy
in the data directory would be generated. Then run
python scripts/run.py --config-name=wanjinyou dataset_name=example case_name=ngp_fox mode=render_path is_continue=true
The synthesized images can be found in ./exp/ngp_fox/novel_images
.
Make sure COLMAP has been installed.
- Run COLMAP SfM:
bash scripts/local_colmap_and_resize.sh <your-data-dir>
Make sure the images are at <your-data-dir>/images
- Generate cameras file:
python scripts/colmap2poses.py --data_dir <your-data-dir>
- Run F2-NeRF!
We provide a script to convert the LLFF camera format to our camera format. For example:
python scripts/llff2poses.py --data_dir=xxx/nerf_llff_data/horns
- Add anti-aliasing
Besides LibTorch, this project is also built upon the following awesome libraries:
- happly for I/O of PLY files
- stb_image for I/O of image files
- tiny-cuda-nn for fast MLP training/inference
- eigen for linear algebra computing
- yaml-cpp for I/O of YAML files
- hydra for configuration
- cnpy for I/O of npy files
Some of the code snippets are inspired from instant-ngp, torch-ngp and ngp-pl.
The COLMAP processing scripts are from multinerf. The example data ngp_fox
is from instant-ngp.
Cite as below if you find this repository is helpful to your project:
@article{wang2023f2nerf,
title={F2-NeRF: Fast Neural Radiance Field Training with Free Camera Trajectories},
author={Wang, Peng and Liu, Yuan and Chen, Zhaoxi and Liu, Lingjie and Liu, Ziwei and Komura, Taku and Theobalt, Christian and Wang, Wenping},
journal={CVPR},
year={2023}
}