[Bug/issue] Why i get grid-like artifacts when train with only a few video clips?

[zz7379]

seq="coco-3"
bash scripts/train.sh lab4d/train.py 6 --seqname $seq --logname base --fg_motion bob --num_rounds 40

normal-coco-3-base-0t-reset-f.mp4

rgb-coco-3-base-0t-reset-f.mp4

(Addition question: is the anchor_x = (0, 4000) right when i set num_rounds to 40 or 120?

[gengshan-y]

Hi sorry for the late reply, this grid artifact happens when there is not sufficient visual evidence to fit the model which might be too flexible. This is expected and there are a few ways if you want to impmrove it:

(1) the most straightforward way is to lower the positional encoding from 10 to 8 or 6 will make the model less flexible but smoother.
(2) another fix is to use the PPR branch, where we adopted the numerical graidents from neuralangelo, which makes eikonal loss more well-behaved and better encourages a valid surface than analytical gradient in the main branch

[zz7379]

Problem solved by your instruction! Thanks gengshan!