Endo-4DGS: Endoscopic Monocular Scene Reconstruction with 4D Gaussian Splatting

Yiming Huang , Beilei Cui , Long Bai *, Ziqi Guo, Mengya Xu, Hongliang Ren

Medical Image Computing and Computer Assisted Intervention (MICCAI) 2024

TODO

EndoNeRF
StereoMIS
Pretrained checkpoints

Environments

We build the Python environment using Anaconda:

git clone https://github.com/lastbasket/Endo-4DGS.git
cd Endo-4DGS
git submodule update --init --recursive
conda create -n ED4DGS python=3.7 
conda activate ED4DGS

pip install -r requirements.txt
pip install -e submodules/depth-diff-gaussian-rasterization
pip install -e submodules/simple-knn
pip install torch==2.0.0 torchvision==0.15.1 torchaudio==2.0.1 --index-url https://download.pytorch.org/whl/cu118

Datasets

We used two datasets for training and evaluation.

EndoNeRF

We used EndoNeRF dataset provided by Yuehao Wang.

StereoMIS

We used StereoMIS dataset from Michel Hayoz

The data structure is as follows:

./data
├── endonerf
│    ├── cutting_tissues_twice
│    │    └── ...
│    ├── pulling_soft_tissues
│    │    └── ...
│    └── ...
└── stereomis
    └── ...

Preparing the Pretrained Depths from Depth-Anything

Download depth_anything_vits14.onnx and place in: ./submodules/depth_anything/weights/depth_anything_vits14.onnx

sh prepare_depth.sh

Training

sh train.sh

More configurations can be found in arguments/$DATASET.py

Evaluation

# Render the testing result
sh render.sh
# Evaluation
sh eval.sh

Acknowledgement

Thanks the authors for their works: StereoMIS, diff-gaussian-rasterization-depth, EndoNeRF, 4DGaussians, Depth-Anything-ONNX.

Divyanshumalik1/Endo-4DGS