Accurate Differential Operators for Hybrid Neural Fields

This repository contains the associated code for the paper titled

Accurate Differential Operators for Hybrid Neural Fields. Aditya Chetan, Guandao Yang, Zichen Wang, Steve Marschner, Bharath Hariharan.

Updates

[2023/12/22] Initial code release.
[2023/12/10] Code release coming soon!

Setup

For setting up the environments required for training the models and running the rendering demo, please follow the steps given in setup.

Experiments

Training

For training your own models:

First activate the conda environment for training using:

conda activate hnf-train

First place your mesh that is normalized such that it lies within the $[-1, 1]^3$ hypercube in the data folder.
Then, create a config using one of the examples shared in the configs folder. In most cases, it should be as simple as replacing the path to the mesh with your own.
Then, run the following command:

python3 train.py configs/<your_config>.yaml

If you want to make any changes to any other hyperparameters from the command line, here is an example of how to do it, shown using the learning rate:

python3 train.py configs/<your_config>.yaml --hparams trainer.opt.lr=0.001

For fine-tuning, follow the same commands as training, except that you need to specify the path to the checkpoint you want to fine-tune from:

python3 train.py configs/<your_config>.yaml --resume --pretrained <path_to_checkpoint>

Rendering

In order to view rendering results:

First activate the conda environment for rendering using:

conda activate hnf-render

Now open the notebook rendering.ipynb and set the kernel to hnf-render.
Select the shape you want in the dropdown and run the cells in order.
Feel free to add your own shapes by training models as described above and adding settings for the shape in the settings_dict variable in the notebook.

Citation

If you found the code in this repository useful, please consider citing our paper:

@misc{chetan2023accurate,
      title={Accurate Differential Operators for Hybrid Neural Fields}, 
      author={Aditya Chetan and Guandao Yang and Zichen Wang and Steve Marschner and Bharath Hariharan},
      year={2023},
      eprint={2312.05984},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Acknowledgements

We thank the authors of torch-ngp, ldif, tiny-cuda-nn for making their code publicly available.

justachetan/hnf-derivatives