Notice: Still reorganizing and consolidating codes...The current repo only contains the training and inference parts for NeRF. Codes for 2D fitting and 3D fitting, together with the CUDA implementations will be released afterwards.
This repository contains the code (in PyTorch Lightning) for the paper:
Neural Fourier Filter Bank
Zhijie Wu, Yuhe Jin, Kwang Moo Yi
CVPR 2023
In this project, we propose to learn a neural field by decomposing the signal both spatially and frequency-wise. We follow the grid-based paradigm for spatial decomposition, but unlike existing work, encourage specific frequencies to be stored in each grid via Fourier feature encodings. We then apply a multi-layer perceptron with sine activations, taking these Fourier encoded features in at appropriate layers so that higher-frequency components are accumulated on top of lower-frequency components sequentially, which we sum up to form the final output. We do the evaluations in the tasks of 2D image fitting, 3D shape reconstruction, and neural radiance fields. All results are tested upon an Nvidia RTX 3090.
If you have any questions, please feel free to contact Zhijie Wu (wzj.micker@gmail.com).
- Python 3.8
- CUDA 11.6
- PyTorch 1.12.0
- PyTorch Lightning
- torch-scatter
- apex
- tiny-cuda-nn
- Install requirements by
pip install -r requirements.txt
Note: Our current implementations are heavily based on the ngp-pl repo. For further details, please also refer to their codebase.
A quickstart:
--root_dir <path/to/lego> --exp_name Lego --num_epochs 30 --lr 2e-2 --eval_lpips --no_save_test
It will train the Lego scene for 30k steps. --no_save_test
is to disable saving synthesized images.
More options can be found in opt.py
and FFB_config.json
under the config
folder.
To compute the metrics for the eight Blender scenes, please run the script benchmark_synthetic_nerf.sh
under the folder benchmarking
.
Codes to be released
Codes to be released
@misc{wu2022neural,
title={Neural Fourier Filter Bank},
author={Zhijie Wu and Yuhe Jin and Kwang Moo Yi},
year={2022},
eprint={2212.01735},
archivePrefix={arXiv}
}
Our codebase is under the MIT License.
- Finish the CUDA version
- Release codes for 2D image fitting & 3D shape reconstruction