Shape of the mask does not match the shape of the indexed tensor

[efyphil]

Hi, thanks for your work, I am experimenting with my custom dataset, and have gone through all the steps that were suggested, namely generating depth maps, and normal maps, they look good. I try to run the algorithm and it even initializes the original discharged point cloud, but then crashes with this error
shape of the mask [21665] at index 0 does not match the shape of the indexed tensor
I have also attached my dataset structure. Classic gs3d algorithms work correctly on it. I have also attached the code that I am trying to run
ns-train dn-splatter --pipeline.model.normal-supervision mono --pipeline.model.use-depth-loss True --pipeline.model.mono-depth-lambda 0.1 --pipeline.model.use-normal-loss True --pipeline.model.use-depth-smooth-loss True coolermap --data ../../../../datasets/chair_nerf/colmap/ --normals-from pretrained --load-normals True

[maturk]

Hi, this is probably fixed in #45,

Inside the root of dn_splatter, run the following commands:

git pull
git checkout maturk/gsplat_migrate
pip install -e .

Make sure you have nerfstudio version >= 1.1.2 and gsplat version >= 1.0.0 installed.

Then you can run your commands.

[efyphil]

AttributeError: 'DNSplatterModel' object has no attribute '_get_background_color'. Did you mean: 'background_color'? Now

Hi again I have updated all versions but now I get this error. I see the same piece of code in splatfacto
I add it into dn_model.py, and now it's work, seems like, testing further
Here's the code

    def _get_background_color(self):
        if self.config.background_color == "random":
            if self.training:
                background = torch.rand(3, device=self.device)
            else:
                background = self.background_color.to(self.device)
        elif self.config.background_color == "white":
            background = torch.ones(3, device=self.device)
        elif self.config.background_color == "black":
            background = torch.zeros(3, device=self.device)
        else:
            raise ValueError(f"Unknown background color {self.config.background_color}")
        return background

[efyphil]

  File "/home/user/.local/lib/python3.10/site-packages/nerfstudio/scripts/train.py", line 100, in train_loop
    trainer.train()
  File "/home/user/.local/lib/python3.10/site-packages/nerfstudio/engine/trainer.py", line 265, in train
    callback.run_callback_at_location(
  File "/home/user/.local/lib/python3.10/site-packages/nerfstudio/engine/callbacks.py", line 115, in run_callback_at_location
    self.run_callback(step=step)
  File "/home/user/.local/lib/python3.10/site-packages/nerfstudio/engine/callbacks.py", line 100, in run_callback
    self.func(*self.args, **self.kwargs, step=step)
  File "/home/user/nerfstudio/dn_splatter/dn_model.py", line 283, in refinement_after
    and self.max_2Dsize is not None

No, another problem, try ro solve it right now

[maturk]

Aha, it seems that Nerfstudio v1.1.2 is actually broken. It is the most latest nerfstudio release, however, there have been important merges upstream that fix issues with Nerfstudio v1.1.2. I have made an issue for this problem in the upstream nerfstudio, however it might take a day or two to get this resolved.

You have two options:

1. use a old version of nerfstudio and gsplat:

pip install nerfstudio==1.1.0
pip install gsplat==0.1.11

and install and run the code in the main branch of dn-splatter using:

cd dn-splatter/
git checkout main
pip install -e .

2. use the maturk/gsplat_migrate branch using gsplat v1.0.0 using the following instructions:

You need to install the most up-to-date nerfstudio version directly from the github repository:

// create conda environment with the right dependencies
conda create --name dn -y python=3.8
conda activate dn
pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 --index-url https://download.pytorch.org/whl/cu118
conda install -c "nvidia/label/cuda-11.8.0" cuda-toolkit

// install gsplat v1.0.0
pip install gsplat==1.0.0

// install nerfstudio
git clone git clone https://github.com/nerfstudio-project/nerfstudio.git
cd nerfstudio
pip install -e .
cd ../

// install dn-splatter
cd dn-splatter/
git checkout maturk/gsplat_migrate
pip install -e .

I will have to wait for a few fixes in nerfstudio to be able to merge my maturk/gsplat_migrate into main. Sorry for the hastle. I suggest using method 1) for now.

Thanks,
maturk

[maturk]

Hi @efyphil, I think this problem is now fully fixed... git checkout main and git pull to get the latest updates.

The dependencies for dn-splatter are now:

nerfstudio version >= 1.1.3 (bumped up from 1.0.3) 
gsplat version >= 1.0.0 (bumped up from 0.1.11)

Feel free to open another issue if you find any bugs.

thanks,
maturk

[efyphil]

Thank you for such good encouragement!