NeuroFluid

Code repository for this paper:
NeuroFluid: Fluid Dynamics Grounding with Particle-Driven Neural Radiance Fields.
Shanyan Guan, Huayu Deng, Yunbo Wang^†, Xiaokang Yang
ICML 2022
[Paper] [Project Page]

Please cite our paper if you find this code useful:

@inproceedings{guan2022neurofluid,
  title={NeuroFluid: Fluid Dynamics Grounding with Particle-Driven Neural Radiance Fields},
  author={Guan, Shanyan and Deng, Huayu and Wang, Yunbo and Yang, Xiaokang},
  booktitle={ICML},
  year={2022}
}

Dependencies

NeuroFluid is implemented and tested on Ubuntu 18.04 with python == 3.7 and cuda == 11.1. To run NeuroFluid, please install dependencies as follows:

Create an environment

conda create -n fluid-env python=3.7
conda activate fluid-env

Install Open3D.

git clone https://github.com/isl-org/Open3D-ML.git && cd Open3D-ML && git checkout c461790869257e851ae7f035585b878df73bc093
pip install open3d==0.15.2
pip install -r requirements.txt
pip install -r requirements-torch-cuda.txt

Install Pytorch3D

conda install -c fvcore -c iopath -c conda-forge fvcore iopath
conda install -c bottler nvidiacub
wget -c https://github.com/facebookresearch/pytorch3d/archive/refs/tags/v0.6.1.tar.gz
tar -xf v0.6.1.tar.gz && cd pytorch3d-0.6.1 && pip install -e .

install other dependencies
```
pip install -r requirements.txt
```

Generate data

See this guide to generate fluid data.

Running the pretrained model

Evaluate NeuroFluid:

python eval_e2e.py --resume_from $MODEL_PATH --dataset DATASET_NAME

DATASET_NAME is one element of [bunny, watercube, watersphere, honeycone]. To compute PSNR/SSIM/LPIPS results, run utils/evaluate_images.ipynb

Evaluate the transition model

python eval_transmodel.py --resume_from $MODEL_PATH

Evaluate the renderer

python eval_renderer.py --resume_from $MODEL_PATH --dataset DATASET_NAME

DATASET_NAME is one element of [bunny, watercube, watersphere, honeycone].

Run the training script

Warm-up stage

CUDA_VISIBLE_DEVICES=3 python train_renderer.py --expdir exps/watercube --expname scale-1.0/warmup --dataset watercube --config configs/watercube_warmup.yaml

End2end stage

CUDA_VISIBLE_DEVICES=0 python train_e2e.py --expdir exps/watercube --expname scale-1.0/e2e --dataset watercube --config configs/watercube_e2e.yaml

Fetch data

Download dataset and models from this link.

Acknowledgement

The implementation of transition model is borrowed from DeepLagrangianFluids. Please consider cite their paper if you use their code snippet:

@inproceedings{Ummenhofer2020Lagrangian,
        title     = {Lagrangian Fluid Simulation with Continuous Convolutions},
        author    = {Benjamin Ummenhofer and Lukas Prantl and Nils Thuerey and Vladlen Koltun},
        booktitle = {International Conference on Learning Representations},
        year      = {2020},
}

We refer to nerf_pl to implement our renderer. Thank D-NeRF for providing the script of computing PSNR/SSIM/LPIPS.

syguan96/NeuroFluid