Oblique-MERF

Installation

Create environment

We recommend using conda to manage dependencies. Make sure to install Conda before proceeding.

conda create --name oblique-merf -y python=3.8
conda activate oblique-merf

Dependencies

Install PyTorch with CUDA (this repo has been tested with CUDA 11.8) and tiny-cuda-nn. cuda-toolkit is required for building tiny-cuda-nn.

pip install torch==2.0.1+cu118 torchvision==0.15.1+cu118 --extra-index-url https://download.pytorch.org/whl/cu118

conda install -c "nvidia/label/cuda-11.8.0" cuda-toolkit
pip install ninja git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch

Install Nerfstudio and Oblique-MERF

cd nerfstudio
pip install -e .

cd ..
pip install -e .

Data preparation

Our data format requirements follow the instant-ngp convention.

Matrix City

To download the Matrix City dataset, visit the official page. You can opt to download the "small city" dataset to test your algorithm. This dataset follows the instant-ngp convention, so no preprocessing is required.

Custom Data

We highly recommend using Metashape to obtain camera poses from multi-view images. Then, use their script to convert camera poses to the COLMAP convention. Alternatively, you can use COLMAP to obtain the camera poses. After obtaining the data in COLMAP format, use ns-process-data to generate the transforms.json file.

ns-process-data images --data path/to/data --skip-colmap

Our loaders expect the following dataset structure in the source path location:

<location>
|---images
|   |---<image 0>
|   |---<image 1>
|   |---...
|---sparse(optionally)
    |---0
        |---cameras.bin
        |---images.bin
        |---points3D.bin
|---transforms.json

Workflow

Training

ns-trian merf --data path/to/data

Evalation

ns-eval --load_config path/to/config --render_output_path path/to/renders --output-path path/to/metrics

Baking

ns-baking --load-config path/to/output/config --baking_config.baking_path path/to/bakings

Real-Time Rendering

Citation

If you use this repo or find the documentation useful for your research, please consider citing:

@inproceedings{zeng2025oblique,
    title={Oblique-MERF: Revisiting and Improving MERF for Oblique Photography},
    author={Zeng, Xiaoyi and Song, Kaiwen and Yang, Leyuan and Deng, Bailin and Zhang, Juyong},
    booktitle={International Conference on 3D Vision},
    year={2025}
}

Ackownledgements

This repository's code is based on nerfstudio, MERF and City-on-Web. We are very grateful for their outstanding work.

USTC3DV/Oblique-MERF