nerfstudio-project/nerfacc

issue in train_ngp_nerf_prop.py

Korace0v0 opened this issue · 0 comments

Hi, I seem to find a bug in examples/utils.py while using train_ngp_nerf_prop.py with 360_v2 dataset.

Simply add print(f'shape of proposal_params: {proposal_network.mlp_base.params.shape}') after sigmas

    def prop_sigma_fn(t_starts, t_ends, proposal_network):
        t_origins = chunk_rays.origins[..., None, :]
        t_dirs = chunk_rays.viewdirs[..., None, :]
        positions = t_origins + t_dirs * (t_starts + t_ends)[..., None] / 2.0
        sigmas = proposal_network(positions)

I get the same parameter shape, which means that I am using the same proposal_network.

shape of proposal_params: torch.Size([862208])
shape of proposal_params: torch.Size([862208])

The correct shape should be like this:

proposal_networks[0].mlp_base.params.shape
torch.Size([768576])
proposal_networks[1].mlp_base.params.shape
torch.Size([862208])

It seems that prop_sigma_fns was not set properly. This will lead to zero grad of the first proposal network.

        t_starts, t_ends = estimator.sampling(
            prop_sigma_fns=[
                lambda *args: prop_sigma_fn(*args, p) for p in proposal_networks
            ],