nerfstudio-project/nerfstudio

`NeuS-facto` generates a weird rendering result on the DTU skull dataset

Closed this issue · 1 comments

barikata1984 commented

Hi, everyone,

Does anyone help me train a neus-facto model on the DTU-65 dataset provided by the sdfstudio project? I run the following code on nerfstudio==1.1.5:

ns-train neus-facto --data path/to/dtu-65

But the visualizer shows something like:
image

As you can see, the skull is rendered on (or within?) a cube-like space. That is totally different from the should-be scene for this dataset. Although this result, similar to the Fifth Dimension space of Interstellar, looks great as an artwork, I want to have the should-be result. Would anyone be able to give me any suggestions?

My hardware/OS setup is:

  • OS: Ubuntu 22.04
  • GPU: RTX 3090
  • CUDA: 12.1
  • PyTorch 2.1.2

Thanks in advance!

barikata1984 commented

Okay, simply neus-facto is hard to tune. With the following command, I got the attached result.

s-train neus-facto --data data/sdfstudio-demo-data/dtu-scan65/ --pipeline.model.sdf-field.geometric-init True --pipeline.model.background-model mlp --pipeline.model.sdf-field.use-grid-feature True --pipeline.model.sdf-field.use-appearance-embedding True --pipeline.model.sdf-field.inside-outside False --pipeline.model.mono-normal-loss-mult 0.01 --pipeline.model.mono-depth-loss-mult 0.01 --pipeline.model.near-plane 0.05 --pipeline.model.far-plane 100 --pipeline.model.overwrite-near-far-plane True --pipeline.model.sdf-field.bias 0.3 --pipeline.model.eikonal-loss-mult 0.01  --pipeline.model.sdf-field.num-layers 1

image