maincold2/Compact-3DGS

Gaussian volume masking and frustum culling.

Opened this issue · 0 comments

Hi guys,

I have a question regarding the Gaussian volume masking. As shown in the lines below, the "mask" and/or the "_pc.mask", which is the Gaussian Volume Mask (I hope I at least get this correct):

mask = ((torch.sigmoid(pc._mask) > 0.01).float()- torch.sigmoid(pc._mask)).detach() + torch.sigmoid(pc._mask)
if rvq_iter:
scales = pc.vq_scale(pc.get_scaling.unsqueeze(0))[0]
rotations = pc.vq_rot(pc.get_rotation.unsqueeze(0))[0]
scales = scales.squeeze()*mask
rotations = rotations.squeeze()
opacity = pc.get_opacity*mask

I am wondering if the "_mask" of some 3G Gaussians outside of the current frustum will also be updated since the frustum culling is done inside the Rasterization function.
So, my question is: will this potentially result in removing Gaussians unexpectedly? Or we can simply rely on the random viewpoint of training iterations (around 3,000) between every mask pruning to avoid that?