I have the depth map but have no surface normal, can I train without surface normal?

[PearsonChen123]

I tried the command:

ns-train dn-splatter
--data [Path to data]
--pipeline.model.use-depth-loss True
--pipeline.model.sensor-depth-lambda 0.2
--pipeline.model.use-depth-smooth-loss True
--pipeline.model.use-normal-loss False
--pipeline.model.random-init True
normal-nerfstudio
--load-pcd-normals False
--load-3D-points False

and turns out:


Traceback (most recent call last):

File "/home/pengcc/miniconda3/envs/nerfstudio/bin/ns-train", line 8, in
sys.exit(entrypoint())
File "/mnt/m/algorithm/nerfstudio_2/nerfstudio/nerfstudio/scripts/train.py", line 262, in entrypoint
main(
File "/mnt/m/algorithm/nerfstudio_2/nerfstudio/nerfstudio/scripts/train.py", line 247, in main
launch(
File "/mnt/m/algorithm/nerfstudio_2/nerfstudio/nerfstudio/scripts/train.py", line 189, in launch
main_func(local_rank=0, world_size=world_size, config=config)
File "/mnt/m/algorithm/nerfstudio_2/nerfstudio/nerfstudio/scripts/train.py", line 100, in train_loop
trainer.train()
File "/mnt/m/algorithm/nerfstudio_2/nerfstudio/nerfstudio/engine/trainer.py", line 237, in train
self._init_viewer_state()
File "/mnt/m/algorithm/nerfstudio_2/nerfstudio/nerfstudio/utils/decorators.py", line 59, in wrapper
ret = func(self, *args, **kwargs)
File "/mnt/m/algorithm/nerfstudio_2/nerfstudio/nerfstudio/engine/trainer.py", line 362, in _init_viewer_state
self.viewer_state.init_scene(
File "/mnt/m/algorithm/nerfstudio_2/nerfstudio/nerfstudio/utils/decorators.py", line 83, in wrapper
ret = func(*args, **kwargs)
File "/mnt/m/algorithm/nerfstudio_2/nerfstudio/nerfstudio/viewer/viewer.py", line 441, in init_scene
image = train_dataset[idx]["image"]
File "/mnt/m/algorithm/nerfstudio_2/nerfstudio/nerfstudio/data/datasets/base_dataset.py", line 152, in getitem
data = self.get_data(image_idx)
File "/mnt/m/algorithm/nerfstudio_2/nerfstudio/nerfstudio/data/datasets/base_dataset.py", line 138, in get_data
metadata = self.get_metadata(data)
File "/mnt/m/algorithm/nerfstudio_2/dn-splatter/dn_splatter/data/dn_dataset.py", line 159, in get_metadata
filepath = self.normal_filenames[data["image_idx"]]
IndexError: list index out of range

I checked the code,

Seems like it will load the surface normal no matter what. How can I train without the surface normal, and use depth supervision only?

[maturk]

Yeah, you can disable loading normals. You can append the --load-normals False flag to your commans. For example, when you are using the normal-nerfstudio dataparser:

ns-train dn-splatter
--data [Path to data]
--pipeline.model.use-depth-loss True
--pipeline.model.sensor-depth-lambda 0.2
--pipeline.model.use-depth-smooth-loss True
--pipeline.model.use-normal-loss False
--pipeline.model.random-init True
normal-nerfstudio
--load-pcd-normals False
--load-3D-points False
--load-normals False

Each dataparser has this option to disable looking/loading any normal maps.

Additionally, if you want to disable predicting/rendering normals all together, which removes the normals also from the viser visualization, then you can use the command --pipeline.model.predict-normals False. Otherwise, you can just set the --pipeline.model.use-normal-loss False command which will still render normals but it wont regularize them. Hope this helps, let me know if you run into any issues.

[PearsonChen123]

It‘s working, thank you so much